Context From First Principles
Chapter 01 of 24

What Context Means

By AI, Software Engineering

Explain this chapter with AI

Copy this prompt into ChatGPT, Claude, Gemini, a local model, or another AI.

Apply this chapter with AI

Copy this prompt into ChatGPT, Claude, Gemini, a local model, or another AI.

Consider a concrete evening. A developer asks a coding agent to fix a failing test. The typed message is eleven words long: “the checkout test fails after my last commit, please fix it.”

Behind those eleven words sits an organisation of information enormously larger than the request. The repository holds hundreds of thousands of lines of code. The version control history holds the last commit and every commit before it. The continuous integration log holds the actual failure output. The open editor holds the file the developer was viewing. The agent’s working directory holds shell history, environment variables, and configuration. Earlier in the session, the developer pasted an error trace, the agent ran two searches, and a linter produced three warnings nobody has read yet.

The model that must now act sees none of this directly. It sees one thing: a single rendered invocation, assembled by the surrounding software, containing whatever that software chose to include. Perhaps 40,000 tokens of instructions, file excerpts, history, and tool definitions. Perhaps 4,000. The difference between those two numbers is not a property of the model. It is a decision made by the system around the model, and it will shape everything the model does next.

This is the situation the book investigates. The word “context” is routinely used to mean six different things: the user’s prompt, the chat transcript, the model’s window size, the retrieved documents, the system’s memory, and sometimes the entire application state. These are not the same object. Until they are separated, no serious engineering is possible, because every measurement and every intervention will be aimed at the wrong layer.

Four things that are not the same thing

Start with the full universe of information. Call it available information: everything the surrounding system could potentially access. Source files, databases, web pages, prior sessions, teammates’ notes, sensor feeds, secrets in a vault. Available information is unbounded in practice. No model invocation will ever contain all of it, and most of it is irrelevant to any given computation.

Narrow to what the system keeps across interactions. Call it session state: information the surrounding software persists from one step to the next, whether or not the model currently sees it. Conversation history on disk, cached tool results, the plan file the agent wrote last turn, the authentication token, the compressed summary of yesterday’s work. Session state survives between invocations. It is the system’s memory of the task so far, and much of it may never be shown to the model again.

Narrow further to what the model actually receives right now. This is context, and the book uses the word in exactly this sense throughout: the information made available to a model for a particular computation. If a file sits on disk but is not included in this invocation, it is available information and possibly session state, but it is not context. If a prior conversation turn is summarised and the summary is included, the summary is context and the original turn is not. Context is what can influence this computation. Everything else cannot, however relevant it might be.

Finally, separate the container from its contents. The context window is capacity: the bounded input budget associated with this model invocation, measured in tokens. OpenAI’s current documentation, for example, describes the context window as covering input, output, and reasoning tokens in a single request, and treats each request as independent and stateless, with multi-turn state re-supplied per call. The window tells us how much the model can receive. It says nothing about which bits deserve to be inside.

In compact form:

available information
        ≠
session state
        ≠
current context
        ≠
context-window capacity

Confusing any two of these produces characteristic errors. Teams that confuse available information with context assume the model “knows” the repository because the repository is mounted; it does not, unless file contents were placed in the invocation. Teams that confuse session state with context assume the model remembers what happened three turns ago; it remembers only what the harness re-supplied, and re-supply is a choice involving truncation, reordering, and summarisation. Teams that confuse context with the context window read an advertised limit of one or two million tokens as permission to include everything retrievable; the chapters ahead show why that reading fails behaviourally even when it fits numerically. And the most stubborn conflation of all is context with the prompt: the user’s message is the most salient item in the bundle and the item the team laboured over, so it receives the attention while the other 95 per cent of the invocation goes unexamined. Prompt engineering skillfully applied to 5 per cent of the input cannot compensate for the remaining 95 per cent assembled carelessly.

A definition to build on

The book’s primitive, stated once and used everywhere:

Context is the information made available to a model for a particular computation.

Three phrases in that sentence carry weight. “Made available” marks context as the product of an assembly process: something selected, ordered, and represented the information. “To a model” marks the boundary of influence: context is the input side of one inference step, not the system’s beliefs or the user’s intentions. “For a particular computation” marks its scope: context is per-invocation. The next invocation may carry a different bundle, and anything that must persist across invocations must be deliberately carried there.

This definition is established background in the sense that it restates, in precise form, how the major providers already describe their interfaces: per-request rendered inputs composed of instructions, tools, history, and media. OpenAI’s prompt-caching documentation, for instance, describes the cacheable unit as the model’s “full rendered context” including provider instructions, developer messages, tool definitions, and conversation history. Anthropic’s engineering writing on context engineering defines context as the set of tokens included when sampling, and the engineering problem as optimising the utility of those tokens under constraints. The book adopts this shared interface reality as its foundation and builds the engineering discipline on top of it.

Capacity, content, representation

Fixing the definition lets us decompose the problem. Every context bundle answers three questions independently:

                 CONTEXT
                    │
        ┌───────────┼───────────┐
        ↓           ↓           ↓
    CAPACITY      CONTENT      REPRESENTATION
    how much?     which bits?  represented how?

Capacity asks how much the invocation may carry: token budgets, output headroom, reasoning allowances. It is set by the model and the runtime, not by the task. Content asks which bits of the available information were admitted: which files, which history, which instructions, which tool results. Representation asks how the admitted bits were represented: raw prose or structured records, full logs or excerpts, exact tool schemas or paraphrases, chronological or grouped ordering.

The point of the decomposition is that each axis can fail independently. A bundle can fit comfortably in capacity yet contain the wrong bits: the file with the bug excluded, the misleading log included. It can contain the right bits in a representation the model mishandles, such as a wall of undifferentiated log text where a small table of failing assertions would do. And it can be excellent on all three axes for this turn and wrong for the next, because the session moved on while the bundle stood still. Capacity failures announce themselves with truncation errors. Content failures arrive silently as confident wrong answers. Representation failures arrive as the model visibly struggling with material a human would have reformatted before handing over. Learning to tell these apart is half the book.

The decomposition also organises everything that follows. Part I of the book (chapters 2 through 7) stays close to observation: measuring the bundle, exposing its hidden layers, treating capacity as a budget, and demonstrating that more content and careless ordering degrade behaviour. The middle parts take the axes in turn: capacity and its architectural extension, then the operations that change content and representation under a budget: removal, compression, fading, externalisation, and recall. The final part governs the whole: authority among conflicting sources, staleness as content decays, assembly as the act of building one bundle for one computation, and evaluation as the proof that the bundle earned its cost. Each part leans on the primitive defined here, which is why the definition had to come first and had to be exact.

Time is the dimension through which the bundle moves. Across a session, context accumulates as tools return observations; it becomes stale as the world changes underneath it; it is replaced when files are re-read; it is compressed when history is summarised; it leaves the window when truncated and sometimes returns later through re-retrieval; it is reordered by harness logic; and it is generated by the agent itself, in the form of plans, notes, and intermediate artefacts that become inputs to later steps. A definition of context that ignores time describes a single photograph of a moving object. This book treats context as the moving object.

Context as an intervention on behaviour

Why insist on per-computation precision? Because the book’s central claim about method follows from it:

Book hypothesis. For a fixed model and task, changing the context bundle while holding everything else constant changes the distribution of behaviours, and that change is measurable.

This is a hypothesis, not a result. It is the proposition the book intends to test, chapter by chapter, with controlled comparisons. It matters because it converts context from a vague background condition into an experimental intervention: something we vary deliberately and measure the consequences of.

The hypothesis needs its counterfactual stated carefully. Model behaviour varies from run to run even with identical inputs, so a single success under bundle B after a failure under bundle A proves nothing. The claim is about distributions: repeated trials under each bundle, compared. This is why the book’s experiments pre-register decision rules and run multiple trials rather than reporting anecdotes. A context intervention that cannot survive repetition is indistinguishable from luck, and luck is abundant in sampled generation. The discipline feels heavy for “just trying a prompt change,” but the heaviness is the point: without it, every observed improvement is suspect and every kept token is superstition.

The hypothesis has a sharp edge. If two different bundles produce indistinguishable behaviour on a task across repeated trials, then for that task the difference between them did not matter, whatever our intuitions about completeness or cleanliness. Conversely, if removing a single instruction collapses performance, that instruction earned its tokens. Nothing in this book survives on the argument that information is “obviously useful.” Usefulness is demonstrated by comparison.

That standard rules out several tempting non-experiments. A smaller prompt is not evidence of lower cost unless cost is measured. A fluent summary is not evidence of preserved information unless information survival is tested. Retrieved text sitting in the bundle is not evidence of improved behaviour unless behaviour is compared against the bundle without it. Each of these confusions will recur in later chapters attached to a specific mechanism; the discipline for resisting them is established here.

The same separation, away from code

The coding agent is this book’s running example, but the separation must hold everywhere. Take a research assistant asked: “does retrieval-augmented generation reduce hallucination rates?” The available information includes the entire literature, the open web, and the user’s private document store. The session state includes the three papers already discussed and the user’s stated preference for recent work. The context for the next invocation is narrower still: the system instructions, the current question, a short excerpt of the prior exchange, and the five passages a retriever returned, two of which contradict each other. The context window is the token budget all of that must fit within, minus room for the answer.

Notice what goes wrong if the layers blur here. If the team believes retrieved documents are “what the model knows,” they will be surprised when the answer cites a passage that was retrieved but truncated before the invocation, or that sat in session state from an earlier turn but was not re-supplied. If they believe session state is context, they will be surprised when the model ignores the user’s stated preference, which the harness stored but the current bundle omitted. Each surprise is a category error with a precise address: something was available, or persisted, but not made available for this computation.

Four confusions and their symptoms

Each conflation from the opening then has a recognisable symptom in practice:

  • Available information mistaken for context. The model is expected to use information it was never shown: the unopened file, the unpasted log, the database row nobody queried. Symptom: “but it had access to the repo” offered as a rebuttal to a failure that no invocation could have avoided.
  • Session state mistaken for context. The model is expected to remember what the system merely stored. Symptom: surprise when a preference, decision, or correction from ten turns ago vanishes, because the harness stopped re-supplying it or a summariser dropped it.
  • Context mistaken for the window. A large advertised limit is read as a quality guarantee. Symptom: bundles grown until they fit rather than until they work, followed by the discovery, documented in later chapters, that behaviour degrades long before the limit binds.
  • The window mistaken for context. Capacity planning substitutes for content selection. Symptom: teams that can quote their model’s token limit to six figures but cannot say how many tokens their own instructions consume.

Diagnosing a context failure therefore begins with locating it: which of the four objects is actually at fault? Most debugging that calls itself prompt engineering is really an unacknowledged search across these layers.

One bundle, eight turns later

Time deserves a concrete illustration, because static diagrams understate it. Return to the checkout-test session. Turn one carries the user message, project file, tool definitions, and one source file: perhaps 9,000 tokens. By turn four, two tool results and a re-read file have joined; the bundle is 22,000 tokens and contains the same project file for the fourth time. At turn six the harness summarises the earliest turns, replacing 8,000 tokens of history with a 900-token condensation nobody verified. At turn eight the agent writes a plan note to its working state, which the harness now includes in every subsequent invocation. The user’s words across all eight turns total a few hundred tokens. The bundle quintupled, changed composition three times, and acquired a self-generated layer, all without any new request from the user.

Every later mechanism in the book is an operation on this moving object: selecting what enters, preserving what must survive, compressing what can lose fidelity, removing what no longer matters, and recovering what left too early. The operations differ, but they share a precondition. They act on the rendered bundle, not on anyone’s impression of it.

Proposed experiment: the paired bundle

The first measurement the book needs is deliberately minimal. Take one task with an externally checkable outcome, for example a failing test that must be made to pass without breaking others. Fix the model, the model version, the decoding settings, and the task. Construct two context bundles: bundle A containing the task description plus the failing file, and bundle B containing the same plus an additional project document that is plausibly relevant but not necessary. Run each bundle multiple times. Record only three things: the outcome, the total rendered input tokens, and whether the agent’s tool calls differed.

Pre-register the decision rule before running: context will be said to have mattered only if the outcome distributions differ beyond what trial-to-trial variation explains, or if the tool-call traces diverge systematically. If neither happens, the finding is that the extra document cost tokens without changing behaviour on this task. That negative finding is as valuable as a positive one; it is the first instance of the book’s method of keeping, revising, or deleting a mechanism only on measured grounds.

Two design details deserve emphasis because later experiments repeat them. First, the model and task are held constant while only the bundle varies. Any comparison that changes the model version, the decoding settings, and the context at once cannot attribute its outcome to context, and this book will reject such confounded comparisons wherever they appear. Second, behaviour is read from externally checkable outcomes and recorded traces, not from the model’s own account of what helped. Asking the model whether the extra document was useful produces fluent testimony, not evidence; running the task with and without it produces evidence. The paired bundle is the smallest design that honours both constraints, which is why it comes first.

Note what this experiment does not require: no retrieval system, no summariser, no cache analysis, no agent architecture. It requires the ability to render two exact bundles and count what happened. That ability is the subject of the next chapter, and it is harder to obtain than it sounds.

What this chapter does not do

This chapter establishes vocabulary, not mechanisms. It does not teach prompt engineering: the art of phrasing instructions is downstream of deciding which instructions are present. It does not teach retrieval or memory: those systems supply candidates for admission, and the admission decision is the context problem. It does not teach agent design: agents are repeat consumers and producers of context, and their architecture is someone else’s subject. Where those neighbouring problems touch context, later chapters will mark the boundary and stop.

The boundaries are worth stating precisely, because the neighbouring books in this series cover the adjacent territory and this book must not re-teach them. Memory answers a different question: what durable information from the past should remain capable of affecting future behaviour? Context asks what the model should receive for this computation now. Memory may supply candidates to context, through the note-taking and external-store mechanisms of later chapters, but the storage, consolidation, and forgetting of durable records belong to the Memory book. Retrieval likewise supplies candidates: given a need, which stored items are worth considering? The ranking, indexing, and embedding machinery behind that question belongs to the Embeddings and future Retrieval books. This book starts where retrieval stops, at the admission decision and everything after it. Agents repeatedly consume context and generate new potential context with every action, and tools are context producers that also consume context through their definitions; both are treated here strictly as sources and sinks at the boundary, never as architectures to be redesigned. Evaluation and inference appear only as the means of judging context interventions and the substrate that executes them. Each neighbour is invoked where the context problem touches it and not one step further.

A reader who takes one thing from this chapter should take the four-way separation. Available information is what exists. Session state is what the system keeps. Context is what this computation receives. The window is how much it may receive. Every subsequent chapter is an investigation of how to choose the third item wisely given the fourth, while the first two press in from outside.

That investigation cannot begin with opinions about the bundle. It must begin by looking at an actual bundle, token by token. Which raises the question the next chapter answers:

How would we know whether changing the context actually mattered?

We would need to see exactly what the model received. Not the user’s eleven words. The whole rendered thing.

References