# The Context Window Is a Budget
# Book: Context From First Principles
# Chapter URL: https://programmer.ie/books/context/04-chapter/
# Prose fingerprint (sha256, normalized): cc9bfa5a98b4e2e2bfaf029dee9d3856e57ef97b0a5d50e51110171a3dea5df2
# Snapshot generated at build time by Hugo. This is the complete rendered prose
# of the chapter; it is not a transcript of the source file.

Chapter 3 ended with many sources and one finite destination. This chapter is about the destination.
Imagine a team running a coding agent on a model with a million-token window. The story is invented, and it is not evidence about how often anything like it happens. Their sessions rarely pass 150,000 tokens, nothing is truncated, and the window, they reason, has settled the matter. Then a long repair session stops unexpectedly. It might have reached a much lower ceiling imposed by the runtime. A request closer to the provider’s limit might instead leave insufficient room for output, while an expensive sequence of requests could exceed the team’s spending budget without approaching either token ceiling. These are distinct failures: violating provider capacity, failing to reserve enough output capacity, crossing a runtime policy limit, and exceeding an economic budget. A session well below a million tokens illustrates the latter two, not necessarily the former two. The advertised window describes only one boundary of the system the team is operating.
This chapter builds that room. Many possible contributions compete for bounded capacity, and the engineering question changes accordingly: not how to fill the window, but what deserves a share of the budget.
What an advertised limit means The confusion starts with the number. A “1M context window” sounds like permission to supply a million tokens of input. Provider documentation says otherwise, in dialects that should not be assumed identical. The first-principles statement is this: every model exposes constraints on how much information can take part in one computation, and how those constraints are counted has to be read from the provider and not assumed.
Two examples show the shape. OpenAI’s conversation-state documentation says the limit covers input, output and reasoning tokens in a single request. Its current flagship models list a 1.05M-token context window and a 128K maximum output, so a request that used the whole window for input would leave nothing for a reply.
Anthropic’s documentation draws a similar boundary with different rules. Everything in the request counts toward the window: the system prompt, every message including tool results, images and documents, and the tool definitions. The output counts too, including extended thinking. Whether thinking blocks from earlier turns stay in later requests depends on the model. Current models list a 1M window and up to 128K output, and some earlier models, such as Sonnet 4.5, have 200K. Overflow is specified: input that alone exceeds the window is rejected, while on recent models input that fits but leaves too little room for the requested output is accepted and may stop mid-generation with an explicit stop reason. Cached prefixes still occupy the window.
DeepSeek’s pricing page lists a 1M context length and a 384K maximum output, with separate cache-hit and cache-miss input prices. That adds a second point taken up below: tokens that occupy identical capacity can be priced very differently.
These are dated facts, verified against provider documentation in September 2026, and they will change. They are here for the structure they show, not for a comparison of vendors. Window figures are not directly interchangeable across models and providers, because tokenisation, output limits, reasoning accounting and request semantics differ. And in each of the dialects above, the headline number is larger than the input allowance, because output, reasoning and anything added to the request draw on the same capacity. The outer wall is real, but nobody lives at it.
Three kinds of constraint With the headline number in its place, the chapter can state its model. Three constraints govern how much context a system should admit. They are different kinds of thing and are not three token ceilings.
Hard capacity is what the provider and model impose. It can be a set of constraints, such as a maximum context, a maximum output, and modality or request-size limits. The chapter often uses one token ceiling for illustration. It is a provider fact, set outside your control, and it changes with model versions.
Usable budget is a runtime’s own policy: the amount it chooses to allocate to admitted input while respecting the provider’s applicable limits and reserving room for output, reasoning where applicable, counting uncertainty and margin. It is not a provider fact. Where a provider uses a shared input-and-output window, the runtime can model this as capacity less reservations; where the provider imposes separate limits or different accounting rules, that subtraction is only an approximation and must not replace provider-specific validation.
Economic and operational budget constrains what the chosen context may cost in money and time. It is multidimensional: dollars and milliseconds cannot be combined into a single token ceiling without an explicit policy. Monetary charges depend on provider pricing, token accounting, output generated and cache treatment; latency also depends on model execution, infrastructure and request shape. Before a call these are estimates or limits, while actual charges and timings can only be reconciled afterwards.
provider constraint → hard capacity runtime policy → usable context budget pricing, latency, cache→ economic and operational constraint A useful accounting picture of the capacity a runtime allocates is below. It is an engineering model for reasoning about allocation, not a description of any provider’s API semantics, and it is a budget classification, not a further taxonomy of context sources like Chapter 3’s.
hard capacity │ ├── admitted input │ ├── standing present across invocations │ ├── task-specific admitted for the current task │ └── accumulated retained from earlier turns │ └── reserved capacity not admitted context ├── output ├── reasoning, where applicable ├── runtime uncertainty └── safety margin The classes describe lifecycle and accounting behaviour, not source or authority. For a per-invocation total, each captured contribution must be assigned to one class under an explicit rule, even when its lifecycle is ambiguous. A project file edited mid-task may cease to be stable, a retrieved file may become retained history, a regenerated summary may replace earlier material, and a tool definition may appear only after registration. Those events can change classifications across invocations without changing origin. The mapping from Chapter 3’s sources is therefore approximate; the raw parts remain available, ambiguous assignments are reported, and the classified byte totals reconcile to the captured representation rather than an assumed provider token total.
Reserved capacity is different in kind from the three classes above it. It is not another layer of context. It is capacity deliberately left unallocated to admitted input.
Each class has its own dynamics, which is why the decomposition earns its place. A standing item may stay unchanged across a session while occupying capacity on every invocation in which it is admitted. Task-specific material comes and goes with the work. In an unmanaged append-only trajectory, accumulated context grows with the interaction until some mechanism limits or transforms it, and real systems truncate, compact, replace and externalise. Whether accumulated context is the class that breaks budgets in real sessions is a hypothesis and not a measurement.
A worked budget: one session, three turns The accounting model becomes concrete with numbers. Consider a repair trajectory for which the provider’s relevant shared capacity is modelled as 200,000 tokens and the harness sets a policy ceiling of 120,000 input tokens. The 80,000-token difference is a conservative planning allowance for output, reasoning where relevant, uncertainty and margin; it is not a claim that every call will generate or reserve precisely 80,000 tokens. This simplified model assumes a shared capacity constraint and does not replace the provider’s separate output and request rules. The figures are synthetic, as in Chapter 2’s capture table. A real budget table must label observed bytes separately from estimated, declared or provider-reported token counts.
One session against two limits. Standing context never changes, accumulated context starts at zero and ends as the largest share, and the usable ceiling is nearly reached while hard capacity still looks abundant. Illustrative synthetic example from this chapter, not an experimental result. Three readings follow from the arithmetic. First, if the standing block of 8,400 tokens is present on all twelve invocations, and there is exactly one invocation per illustrated turn, it accounts for 100,800 cumulative input-token positions. That is an assumption about the synthetic trajectory, not a conclusion supported by the three plotted snapshots. It is not a bill: caching may change the price. Second, accumulated context reaches 61,300 of 113,800 input tokens by turn twelve, the majority, so a policy that counts only newly admitted material would miss retained occupancy. Third, turn twelve uses approximately 94.8 per cent of the 120,000-token policy ceiling while using 56.9 per cent of the 200,000-token hard capacity. Monitoring only the provider headline misses proximity to the harness’s own limit. The observer of Chapter 2 provides captured material for estimating these quantities; its pre-provider counts must not be presented as provider-verified token usage.
The illustration does not say turn twelve behaved badly. Behaviour is Chapter 5’s subject. It shows only that the accounting has a shape, and that a team tracking only the headline number would not see it.
Inspect this in the Context Viewer The viewer scene for this chapter renders the turn-twelve state of the same synthetic budget from the same numbers: standing 8,400, task 44,100, accumulated 61,300 against the 120,000 policy ceiling, with the 200,000 hard capacity for context. Compare the timeline across turns 1, 6 and 12 in the interactive view. What to examine: which class grows, which ceiling binds first, and how much slack each ceiling leaves. This is a constructed illustration, not a measurement; it establishes the accounting shape and nothing about behaviour or cost in money.
The same synthetic budget in the Context Viewer: turn twelve against the policy ceiling. Constructed illustration from this chapter’s numbers, not a measurement. Open the budget scene in the Context Viewer
Standing and marginal cost The accounting picture implies a practical distinction between what context costs to have and what it costs to add. These describe how admitted material participates over time. They are not fixed types of information.
A standing cost is incurred on every invocation in which an item is admitted, for as long as it stays. In this chapter and later, standing cost names recurring capacity occupancy unless a monetary or latency cost is explicitly named. Tool schemas, project instruction files and stable system material are typical. A tool definition of 500 tokens is small in isolation, and across forty invocations it occupies 20,000 input-token positions. An item has several costs at once: its source size, its per-invocation occupancy, its cumulative exposure across the trajectory, its price, and any cache treatment. These are different measurements, and cumulative occupancy is not the same as cost in money.
A marginal cost is the change in an invocation’s occupancy attributable to an admission or replacement decision: another file read, another tool result, another turn or another retrieved document. It is not always equal to the size of the new item. A new summary might replace a larger history, making the net change negative, while a tool result may displace something else under a ceiling. To report marginal occupancy, compare the relevant before-and-after representations and state whether the figure is gross addition or net change. Marginal costs feel like the real spending because an admission is a visible decision; recurring occupancy is easier to overlook.
An item moves between the two:
new admission → marginal cost now → retained across turns → recurring occupancy later A 10,000-token file loaded once for the task it serves is expensive and possibly justified. The same file re-sent unchanged for thirty turns behaves, after its first turn, like a standing cost. The instrument from Chapter 2 helps tell these apart: repetition and byte stability, derived from consecutive captures, separate recurring occupancy from new material. This connects to Chapter 1’s dimension of time. Cache-aware layout (Chapter 9), tool context (Chapter 17) and pruning and externalisation (Chapters 10 and 13) each inherit this vocabulary.
Per-source accounting A budget nobody tracks is a wish. The desired enforcement ledger attributes each admitted contribution to its source and accounts for its capacity occupancy without double-counting. The current instrument supports a narrower ledger: it records tool definitions, tool results, message roles and compaction requests, but it cannot separate vendor, product and project authorship within an undifferentiated system block. Project-file origin may be declared by an operator rather than observed. Nor can the host-side observer assign an exact provider token count to every source merely because it has captured the source’s bytes. A useful ledger therefore reconciles what it can, labels estimated or declared token allocations, and preserves unattributed material instead of assigning it an invented owner. Chapter 2’s observed, derived, declared, joined and unobserved statuses apply to individual fields, not necessarily entire rows.
Attribution makes ownership and remediation discussable, though it does not fix one owner per line. A project file may belong to repository authors, a tool schema to another team and history policy to the harness. As an engineering model, a standing-cost overrun and an accumulated-context overrun tend to call for different remedies, such as removing a tool or shortening a preamble in the first case and pruning, externalising or compacting in the second, and treating both as “the context got big” tends to prescribe the wrong one. When later chapters propose mechanisms, each should name the ledger lines it moves.
Why systems leave the window unfilled If hard capacity is the outer wall, why not build right up to it? Because several claimants on the capacity cannot be measured in advance, and a runtime reserves against them as a matter of policy:
Output. The response must go somewhere. The provider’s maximum output is one thing and the output a task is expected to need is another. A single invocation must leave room for the response it requests; a multi-call agent also needs a trajectory policy for future invocations and their tool results. Those future calls are not automatically part of the present invocation’s window. Reasoning. Where the model reasons before answering, whether and how that draws on the window is provider-specific, as the accounting above shows. Runtime-added material. Material a harness adds before the observation boundary can be observed and counted. Material added after it, or by the provider, is unobserved, and a reserve is a way to cover it. Anthropic’s context-awareness feature, on the models that have it, injects remaining-capacity information into the request, so the request contains the feature’s own output. Counting error. A count is declared, estimated or provider-reported (Chapter 2), and a bundle measured near the ceiling under one count may overflow under another. Future turns. This is a trajectory policy, not a provider requirement of the current request. Retained history may occupy later invocations, so filling the current request’s budget can force earlier compaction, truncation or externalisation on subsequent turns. A future turn has its own request accounting; its tokens are not literally reserved inside the current invocation. Safety margin. Providers reject or halt at the boundary with varying grace, and margin turns a cliff into a slope. Its size is a policy choice. “What fraction of the window should we use?” has no universal answer. The right reservation depends on the task, expected output, trajectory length, model, reasoning mode, provider semantics and uncertainty. There is no universal utilisation target, and that is why the ceiling is an engineering decision and not a configuration default.
Token count is not cost Four things get called cost, and they are different measurements: token count, capacity occupancy, monetary price and latency. Two bundles with equal token counts can differ in price, since cached prefixes are billed differently from uncached tokens on providers that offer caching, and latency depends on the model, infrastructure, request shape and caching, so it does not follow price. None of this changes what fits in the window. All of it changes what the window costs to use.
Cached tokens are not free capacity. Anthropic’s documentation says cached prefixes still occupy the context window: caching changes what you pay for the tokens, not whether they count. Capacity and money are governed by different rules, and optimising one while ignoring the other produces familiar twin failures, the bundle that fits but is expensive, and the bundle economised into uselessness. The mechanics of caching belong to Chapter 9. Here it is enough that a decision recorded against one constraint must be checked against the others.
The allocation problem The chapter’s main move can now be stated:
The engineering problem is not how to fill the window, but what deserves a share of the budget.
Once many candidate contributions exist, capacity is bounded and the usable budget is a policy, admission is an allocation problem. Admitting an item consumes budget, and that holds whatever any particular provider’s window size turns out to be:
candidate information → admission decision → representation → capacity consumed It is tempting to formalise this as value per token: each candidate carries a value and a token price, and assembly maximises value within the usable budget. The sketch is a useful way of thinking and a dangerous blueprint, because candidates are not independent. Take two files where the second defines the terms the first uses. Admitted together they are worth more than the sum of their separate values, and admitted separately the first may be near worthless. No per-item score survives that dependency. Chapter 22 shows why ranking cannot solve it, and Chapter 23 builds the machinery that treats dependencies as first-class inputs.
What this chapter establishes is that the allocation problem is unavoidable. Every source from Chapter 3’s taxonomy is a candidate claimant, admission is bounded by provider constraints and runtime policy, and the claimants can interact. Filling the window to the brim is not thoroughness. It lets every claimant take what it wants and hopes the result works.
What is tested, and what is not The chapter’s claim is accounting, and its test is accounting, not behaviour. The first check is exact reconciliation of classified bytes to the observer’s captured representation, with unknown categories retained. The second is whether repeated, newly admitted and replaced contributions can be distinguished across captured invocations without mistaking byte identity for identical provider tokenisation. The third is whether a trajectory projection, with a stated growth estimator and observation interval, signals a possible policy-ceiling crossing; it is a conditional projection, not a forecast of how the agent will behave. Finally, declared or estimated token totals must be compared with joined provider usage on correctly matched calls before they are trusted for admission control. Provider totals need not be decomposable into the observer’s categories. Compaction, tool loops and task changes can invalidate a projection, so no single observed growth rate should be promoted into a general rule.
Adding irrelevant material to see whether it helps or harms is Chapter 5’s experiment and not this one. If behaviour is recorded here, it is a covariate for reading the accounting, never the verdict. Nothing in this chapter has been run, and the real-session composition it points at depends on a corpus that does not yet exist.
The chapter’s claim is small and can be stated exactly. Capacity remaining is not evidence value remaining. A window with much unused capacity does not imply that more should be admitted, and so:
can fit ≠ should admit ≠ will help
The next chapter takes the last inequality and asks whether more context, admitted within the limit, is harmless.
References Anthropic. “Context windows.” Official documentation, verified September 2026. Window accounting; sizes by model; thinking and tool use counted; overflow behaviour; cached prefixes occupy the window; context awareness on some models. Provider-specific and version-sensitive. https://platform.claude.com/docs/en/build-with-claude/context-windows OpenAI. “Models.” Official documentation, verified September 2026. Current flagship models: 1.05M context window, 128K maximum output. Provider-specific and version-sensitive. https://developers.openai.com/api/docs/models OpenAI. “Conversation state.” Official documentation, verified September 2026. Token limit includes input, output and reasoning tokens. Provider-specific. https://developers.openai.com/api/docs/guides/conversation-state DeepSeek. “Models & Pricing.” Official documentation, verified September 2026. 1M context length, 384K maximum output, cache-hit and cache-miss input pricing. Provider-specific and version-sensitive. https://api-docs.deepseek.com/quick_start/pricing