Assemble for the Task

The usable input budget is 4,000 tokens. The non-negotiable pile reads: mandatory instructions 1,100, exact constraints 1,400, required tool schema 900, minimum task evidence 1,200. The total is 4,600 against a 4,000-token ceiling, and no ranking, weighting, or cleverness changes the arithmetic. A relevance packer would silently delete 400 tokens of whichever item scored lowest — perhaps the second half of the exact-constraint block, perhaps the tool schema’s required parameters — and emit a plausible, policy-violating bundle whose failure surfaces three turns later as an invalid call or a guessed identifier. The correct output of this computation is not a bundle at all. It is a failure notice naming the 600-token excess — because deleting exact constraints to fit a budget is not optimisation. It is fabrication with good formatting. Everything is eligible; nothing is assembled; and the first question is whether assembly is even possible.

Compilation, not ranking

The book defines context as the information made available to a model for a particular computation. This chapter produces the concrete object satisfying that definition:

The ContextBundle is the exact ordered representation actually supplied to this computation.

Not everything available, retrieved, eligible, relevant, or fitting. The exact bytes in the exact order. And the mechanism producing it:

Context compilation converts a request, a candidate set, explicit policies, and a finite budget into an exact model-visible bundle or an explicit failure.

Request plus candidates plus policies plus budget compile to bundle plus decision trace — or to compile failure. The vocabulary is software-shaped because the contract is software-shaped; no class diagram is being smuggled into the prose. What matters is that selection is only one compiler pass among several. Membership asks which items survive; compilation additionally decides which representation of each item, at which fidelity, as resident payload or reference, with which dependencies accompanying it, placed where, with how much budget remaining, and whether the result is legal at all. Selection sits inside assembly the way parsing sits inside compilation: necessary, prior, and nowhere near sufficient. Any design that scores everything, sorts descending, and fills tokens to the brim has erased most of the architecture the preceding fifteen chapters earned, and this chapter exists to prevent exactly that erasure.

Feasibility before optimisation

Before asking for the best bundle, the compiler asks whether any legal bundle exists. The opening arithmetic is the general case: mandatory standing context plus exact requirements plus required capabilities plus minimum evidence exceeding the usable budget means no legal bundle fits, and the only honest outputs name the infeasibility — insufficient budget, unsatisfied dependency, no legal representation, unresolved required conflict, required source unavailable or of unknown freshness. The failure vocabulary stays small on purpose; what matters is the principle it serves:

When policy cannot construct a legal bundle, the compiler reports that fact rather than fabricating a plausible one.

Truncating instructions, dropping exact identifiers, or silently compacting pinned content to force a fit are not optimisation strategies. They are policy violations with tidy output. Recovery — a larger window, a smaller task, a leaner tool surface, externalisation, human intervention — happens outside the compiler, which never buys feasibility by breaking its own rules. The same discipline governs the opposite end. The context window is a ceiling, never a target: a 4,000-token usable budget may legitimately emit 3,200 tokens when nothing further earns admission, and unused budget is not waste. Chapter 5 established that larger windows must not force larger contexts; the compiler is where the principle executes.

Budgets arrive pre-decomposed from Chapter 4, consumed rather than redefined: hard capacity against usable budget against economic budget, with standing context, task context, accumulated context, and reserved headroom as the standing vocabulary. The allocatable candidate budget is usable input minus mandatory standing context minus reserved headroom — approximate wherever provider accounting differs, exact in shape everywhere. Work it once on the opening numbers: 4,000 usable tokens with 900 of standing instructions and schemas plus 500 reserved for response and tool continuation leaves 2,600 allocatable for candidates — against which the 3,500-token evidence pile already fails before a single discretionary choice is made. The arithmetic is the point: feasibility is visible before ranking begins, and no candidate’s score can change what subtraction already decided. Output and reasoning reserves are subtracted before candidate packing begins, never discovered missing after the input fills the nominal window: assembly consumes an explicit input budget and owns nothing beyond it. Token costs follow the runtime tokenizer where one exists and bytes, characters, plus labelled estimates where none does, with no tokenizer’s count ever presented as universal.

Hard gates, then judgement

The pipeline runs legality before desirability, in approximately this order: candidate inventory; hard eligibility and governance; validation and revalidation; legal representation alternatives; dependency closure; budgeted admission; ordering and layout; bundle validation — emitting bundle plus trace, or failure. Approximately, because the report records whatever ordering the completed argument implies; the rule it obeys is the durable one. Walk one candidate through it to fix the mechanics: an incident artifact arrives in inventory with full, compact, and reference forms; governance admits it as in-scope, authorised data with a floor of compact; revalidation confirms its source version current; alternatives enumerate all three legal forms; closure adds the resolver definition to the reference form’s cost; budgeted admission selects compact after the full form loses on marginal cost; ordering places it with grouped evidence; validation confirms floors, closure, scope, freshness, and fit. Eight stages, each with a recorded reason, none of them a score.

Decisions expressible as hard constraints happen before soft optimisation.

Wrong scope without transfer, insufficient instruction authority, unresolved required freshness, exactness floors, illegal fidelities, privacy boundaries — none of these become negative weights inside a relevance score. An illegal candidate must not turn legal at relevance 0.99. Governance is not a penalty term, and the weighted sum is refused as canonical architecture for exactly that reason: any score blending relevance with authority, freshness, and scope lets a sufficiently relevant out-of-scope item outrank a moderately relevant legal one the moment weights drift. Relevance survives, but inside legal bands — mandatory, required, supported, discretionary, do-not-admit — ordered lexicographically so lower bands never displace higher contractual requirements by arithmetic, with relevance, coverage, cost, and diversity ordering candidates within bands. Weighted scoring stays available as a pre-registered challenger rather than a forbidden technique; the claim is only that one scalar cannot be the policy.

External selection research is consumed at exactly this altitude. Budget-aware subset selection under strict token budgets is a legitimate constrained problem — a recent clinical-text study formulates it knapsack-style with a submodular relevance-coverage-diversity objective and finds the best strategy varies by task and evaluation regime rather than crowning a universal selector. The chapter takes the constrained-subset framing and the no-universal-winner finding, nothing clinical, no objective function. Knapsack itself covers one subproblem and misleads as a whole: value and cost under a budget says nothing about hard eligibility, multiple representations of one source, all-of dependencies, one-of alternatives, authority, freshness, ordering, standing context, or resolver capabilities — and no complexity-theoretic claim is made without a formal problem to attach it to. A recent heterogeneous-agent selector pooling turns, memory, and tool outputs into one relevance-ranked assembly pool is cited as a preprint for the direction of travel — pooled assembly across sources rather than per-source handling — with its objective left unadopted and its effect sizes untouched. The broad field survey stays out of the bibliography: as background mapping it adds nothing the reader-facing argument needs. Anthropic’s engineering guidance extends its long-standing entry with the sentence this chapter actually uses — curate minimal high-signal context iteratively per task, with minimal explicitly not meaning short.

One source, several legal forms

A single content identity routinely offers several admittable forms — full text at six thousand tokens, dense at eighteen hundred, compact at six hundred, anchor at ninety, reference at twenty — of which the task’s floors and ranges legalise a subset. The compiler selects one legal active form per need, not several redundant copies, and therefore needs to know that full and compact incident-17 are alternatives sharing one identity rather than two candidates that happen to overlap. Source identity against representation identity, inherited from Chapter 18, is what prevents double-spending the budget on the same episode twice. Alternatives are usually one-of, but the anchor-plus-reference composite earned in Chapter 13 survives as an allowed combination where the fixture requires it: resident meaning plus recoverability in one admission, represented explicitly rather than discovered by accident.

Dependencies make local token counts lie, which is the chapter’s major costing insight. A twenty-token artifact reference is not a twenty-token admission when usefulness requires a 650-token resolver definition, provenance metadata, delegation records, or continuation markers alongside it:

A candidate’s local token count may differ from its marginal bundle cost.

Dependency closure names the difference: the additional context that must accompany a selected candidate for it to remain usable and policy-compliant, drawn only from dependencies earlier chapters demonstrated. Shared dependencies then break naive ranking a second time — ten references splitting one resolver definition pay it once, so marginal cost moves with the already-selected bundle and static value-per-token ordering misfires. The worked case is deliberately small: candidate A with high value at 100 tokens plus a 900-token dependency against candidate B slightly less valuable at 500 tokens with none. Local ratios choose A at apparent 100; bundle-aware costing chooses B at true 500 against 1,000. Representation choice moves the same ledger — full artifact needing no resolver against anchor-plus-reference needing one, with future expansion latency attached — so total compile cost is measured per alternative, never representation tokens alone. Tool definitions enter here as first-class dependencies: an admitted reference the runtime must materialise later requires its resolver capability present in the relevant form, which finally integrates Chapter 17’s standing cost with information selection instead of treating them as separate taxes. Current-computation costs optimise under explicit known prices; long-horizon expected-cost speculation stays telemetry, and no learned utility model is built — the book’s inspectable rules get their chance to prove sufficient before any scorer learns over them.

Groups travel together or not at all. Claim plus qualification, current value plus source identity, both sides of an unresolved disagreement with its conflict status: admitting half of a required group misleads, so all-or-nothing semantics bind where fixtures declare them, sparingly rather than per paragraph. Mandatory context forms the smallest non-negotiable class — governing constraints, the task instruction itself, exact requirements, required safety metadata — counted before any discretionary admission, overflowing loudly rather than trimming silently. Loud overflow names its remedies without executing them, because each belongs to a different owner: a larger window is a provider and budget decision; a smaller task is a task-design decision for whoever framed the request; a leaner tool surface revisits Chapter 17’s capability trade; externalisation of evidence revisits Chapter 13; human intervention is escalation with the diagnostic attached. The compiler’s job ends at naming the excess and its blocking constraints. Anything that resolves infeasibility by quietly violating policy is not recovery — it is the fabrication the failure vocabulary exists to prevent.

Spend, order, render, validate

Discretionary admission runs on staged policy with coverage awareness: three strong candidates explaining one constraint lose to one strong candidate plus a slightly weaker one covering a missing dependency, which is where budget-aware selection’s coverage concepts earn their narrow keep without becoming a universal diversity score. Make the trap concrete: three near-identical explanations of the migration constraint score 0.91, 0.89, and 0.87 on relevance while the resolver-capability note for the artifact reference scores 0.62 — and pure ranking spends the budget triplicating the constraint while the reference it did admit arrives unusable. Coverage-aware staging admits one explanation plus the resolver note and stops, not because diversity outranks relevance but because the second and third explanations add nothing the first did not already supply. Redundancy removal obeys Chapter 10 — exact duplicates may go, while similar-but-independent corroboration, conflicting evidence, and differing provenance stay, because overlap is not redundancy. Marginal utility is always bundle-relative and never claimed as a true estimator: value given the current bundle guides ordering, while the hidden evaluator ledger — must, should, optional, distractor, harmful, visible to evaluation and sealed from assembly — preserves the production reality that compilers act on observable signals while judges hold ground truth. Oracle labels never leak into policy, stated in the experiment spec twice because it matters twice.

Explicit task requirements bypass soft ranking entirely: a task naming two decision records admits them as requirements rather than competitors, and a required-but-absent source returns source-unavailable instead of accepting a similar substitute. Absence stays distinguishable from selection failure. Ordering follows membership under a frozen deterministic layout — governing instructions, task, active state, grouped evidence, capabilities — chosen from earned constraints rather than asserted as canonical, with sensitivity tests subordinate. The sequence earns its order: governance first because nothing below it may contradict what bounds the computation; the task next because every evidence item is interpreted against it; active state before evidence because the model’s own working commitments frame what follows; capabilities last because tool surfaces change most often and Chapter 9 prefers churn at the suffix. None of this is asserted as the universal layout — it is the experiment’s fixed layout, documented and held constant so membership and representation effects stay interpretable. Cache-aware placement stays subordinate too: stable prefixes and dynamic suffixes where behaviourally neutral, correctness first, always, with divergence and mutation telemetry recorded downstream rather than reopened. The rendered bundle carries an order-sensitive identity digest, since swapped orders are different bundles, under the existing privacy rules for identifiers.

Render cost is authoritative. Estimates guide assembly; exact serialisation decides it, and overruns repair by rerunning policy — remove the lowest-priority legal discretionary unit, rerender, repeat — until fit or failure. Byte-slicing the final string is never a budget mechanism. Post-compile validation checks rendered size, mandatory presence, eligibility, floors, dependency closure, scope, freshness, governance metadata, and ordering before anything emits; failure returns diagnostics, reproducibly — same inputs, same code, same blocking constraints with computable excess — never a free-form model apology as the only record. The staged policy claims no optimality even in victory over top-k: it is a baseline architecture, and the oracle exists to display the remaining gap honestly. Oracle compactness measures perfect knowledge, not production targets; extra compiler tokens may be uncertainty’s price, stated plainly rather than apologised for. Consider what that price buys on a typical fixture: the oracle omits the incident anchor entirely, knowing the hidden probe never touches it, while the staged compiler admits anchor plus reference for ninety tokens — hedging a plausible evidence need it cannot rule out. If the probe stays away, those ninety tokens read as waste against the oracle; had the probe turned, they would have read as foresight. The gap report distinguishes the two cases so uncertainty spending is visible as policy, never mistaken for sloppiness.

Proposed experiments

The fixture is deliberately heterogeneous — task instruction, project constraint, retrieved document, memory candidate, tool definition, tool result, agent hypothesis, external artifact, multi-level representations, fresh and stale versions, out-of-scope material, a factual conflict pair, duplicates, a resolver-dependent reference — with every item testing a known assembly interaction and nothing included for checklist completeness. Candidate generation freezes across conditions so failures attribute to assembly; the pool deliberately contains bad candidates, including a minimal-sufficiency oracle representation per task where knowable, enabling over- and under-representation measurement without rerunning Chapter 12. Budgets run tight, medium, and roomy, all pre-registered. Seven traps guard the honest outcomes: the dependency closure that prices a cheap reference honestly; the qualification pair that punishes half-admission; the preserved conflict that must not resolve by price; the stale-cheap against fresh-expensive pair that tests whether Chapter 20 survives assembly; the authoritative, fresh, well-structured out-of-scope candidate that tests whether Chapter 21 does; the mandatory-overflow request whose only correct output is compile failure — the most important negative control in the chapter; and the roomy-slack request where filling the budget with extras is the failure, proving the ceiling is not a target.

Experiment 1 — assembly-policy ladder, over fixed pools, tasks, renderer, and model-independent evaluation: dump-with-truncation as the naive capacity baseline under identical usable budget; relevance top-k without governance; pre-registered weighted packer as the honest scalar challenger; hard-gated greedy testing what legality alone earns; the deterministic staged compiler with alternatives, dependencies, groups, staged admission, deterministic ordering, and final validation; the hidden-label oracle as ceiling; an optional model-assisted selector strictly subordinate, judged on cost, variance, auditability, and violations. Measurements stay dimensional: budget compliance, compile success and failure correctness, must and should recall, precision and distractor admission, harmful and illegal admissions, scope, freshness, authority, floor, dependency, and group-damage violations, rendered tokens, slack — no compiler-quality score, with oracle gaps reported as extra tokens spent and necessary evidence missed. Regret stays descriptive — under- and over-admission, over-fidelity, dependency overcost, avoidable metadata — because Chapter 23 owns behavioural value and this chapter owns bundle quality. A smoke reader may verify rendering and nothing more.

Experiment 2 — component ablations removes one exercised mechanism at a time from the deterministic compiler — scope gate, freshness gate, representation alternatives, dependency costing, group handling, redundancy handling — measuring which failures return. Bundle legality, coverage, and budget are the verdicts; behavioural conclusions wait for the next chapter, preserving the uncertainty that a better bundle may still not read better.

The trace is the second output and gets equal specification. Inclusion and exclusion each carry reasons — availability, eligibility, freshness validation, representation options, dependency closure, priority band, marginal cost, budget state, selected form, final position — with rejected candidates retaining their supporting evidence so scope and freshness rejections stay auditable. Read one exclusion the way an auditor would: the Project B config was available in inventory, failed project eligibility against the active world with no transfer rule, and was rejected before relevance scoring with three tokens of trace recording the mismatch — never admitted, never scored, never silently dropped. Policy versions ride along for replay, policy changes replay the full suite before claiming repair, and local repairs carry the explicit warning that promoting one missed candidate class can flood others: assembly policy is evaluated across the whole fixture set or not at all. Feasibility degrades monotonically into explicit failure as budgets shrink; bundle contents need not nest across budgets, since richer representations may replace several compact ones — legality, policy, and traceability are the invariants, not supersets.

What the laboratory receives

The conceptual runtime interface consolidates at last: request, candidate array, budget, and policy versions in; bundle plus trace, or compile failure, out. Requests carry task identity, target runtime, usable budget, active scope, temporal target, and policy versions — consumed fields only, no universal ontology. Candidates stay on the common interface across retrieval, memory, tools, artifacts, agent state, and conversation, with source behaviour in metadata and adapters. Policies stay separable and explicit — authority, freshness, scope, retention, assembly, render — versioned rather than dissolved into prose or weights. Decision trace and evaluation observation stay on opposite sides of the model call by mandate: the first explains the build, the second records what happened after, and no outcome knowledge leaks backwards. Compiler runs stay pure with respect to content — selecting among approved variants, never rewriting prose mid-pass, with missing-variant pressure resolving to failure or an explicit governed transformation stage, never silent summarisation. Learning from evaluation means a new policy version and a new backtested run, never in-place mutation.

Falsification prefers the boring compiler wherever earned: hard gates plus greedy matching the staged design, representation alternatives adding nothing, dependency costing never changing choices, weighted packing without violations, full bundles fitting safely, failures never occurring at realistic budgets, extra stages not closing the oracle gap. Negative results delete machinery by the standing rule — simple compilers kept, dependency engines unbuilt, single representations retained, staged costs reconsidered. The recommended next step is implementation before further drafting: the contract above is precise enough to freeze, and a smallest deterministic synthetic compiler in Project Context would let the evaluation chapter judge real frozen bundles instead of proposing evaluations. The preview, documentation only: a future stage that, given a versioned synthetic pool, policy set, and budget, deterministically emits either a valid bundle with complete trace or an explicit failure. No code begins here. What remains is proving the bundle helped — whether the assembled, legal, budgeted, traced context actually changes behaviour for the better, which is a different chapter’s entire job.

References

• Qureshi, K., Martin, G., Peng, Y. “Budget-Aware Routing for Long Clinical Text.” Peer-reviewed, Findings of ACL 2026. Knapsack-constrained subset selection with a submodular relevance-coverage-diversity objective; routing by budget regime with task-dependent optima. Used for the constrained-subset framing and the no-universal-selector finding; clinical objectives never imported. https://aclanthology.org/2026.findings-acl.2114/
• Ghulyani, M., Singh, A., Bharadwaj, K., et al. “PACMS: Submodular Context Selection as a Pluggable Engine for LLM Agents.” Preprint, arXiv:2606.20047 v2, September 2026. Unified turns/memory/tool-output pool with facility-location selection against recency truncation. Cited for the pooled-assembly direction; objective unadopted, effect sizes untouched. https://arxiv.org/abs/2606.20047
• Anthropic Applied AI team. “Effective context engineering for AI agents.” First-party engineering essay, September 2025, verified September 2026. Iterative minimal high-signal curation with minimal explicitly not meaning short. https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents