# Assemble for the Task
# Book: Context From First Principles
# Chapter URL: https://programmer.ie/books/context/22-chapter/
# Prose fingerprint (sha256, normalized): 241d26cc1049437a67a85b72c83e34d26d7597ef85dd0fc73279f72d6a9d452a
# Snapshot generated at build time by Hugo. This is the complete rendered prose
# of the chapter; it is not a transcript of the source file.

The usable input budget is 4,000 tokens. The material that must be present reads: standing instructions, 1,100 tokens; exact constraints, 1,400; the schema of a tool the task cannot proceed without, 900; the minimum evidence for the task, 1,200. That is 4,600 tokens against a ceiling of 4,000, and no ranking, weighting or cleverness changes the arithmetic:
For this fixture, the 4,000-token usable input budget includes the standing-instruction allocation. It has not already been deducted as a fixed envelope cost.
mandatory cost 4,600 > budget 4,000 → no valid bundle (excess 600) A naïve relevance packer might try to fit the request by omitting whichever material scores lowest, perhaps part of the exact-constraint block or required tool parameters. That produces a plausible-looking bundle whose declared obligations have been violated. An optimiser that enforces those obligations must instead report infeasibility, identifying the violated budget and the mandatory material. The later invalid call or guessed identifier is a possible consequence of silent omission, not an observed outcome of this illustrative fixture.
The objection is not to optimisation. An optimiser can legitimately return infeasible. The failure is optimising an objective while treating required constraints as soft preferences. Everything in that pile was eligible and nothing could be assembled, so the first question, before any question about which candidates are best, is whether assembly is possible at all. The chapter’s question is:
Given candidates that may participate in a computation, what constraints decide whether a valid bundle can be assembled, and which eligible material should occupy the finite budget?
The chapter owns allocation under a finite budget, feasibility, mandatory versus optional material, bundle-relative cost, dependencies, coupled material, representation alternatives, stopping and structured failure. It does not own retrieval and candidate generation (Chapter 14), retention (7), compression (11 and 12), references (13), tools (17), representation semantics (18), authority, freshness or scope (19 to 21), ordering effects (6) or the implementation (23).
The objects and where this chapter sits The word context covers several objects, and this chapter needs them distinct. The diagram is a logical account of responsibility, not a claim that recovery always precedes eligibility or that ordering can be postponed without affecting cost. Candidate generation may yield metadata-only references; recovery may occur after preliminary selection, and exact rendered feasibility can require reconsidering both selection and order:
available information → candidate generation and recovery ([Chapter 14](/books/context/14-chapter/)) → candidates with eligibility judgements (Chapters 19 to 21) → eligible candidate pool → allocation: this chapter chooses the selected set → ordering ([Chapter 6](/books/context/06-chapter/)) → rendering → ContextBundle Chapter 14 uses admission for requesting and recovering candidate representations, and says admission is not budgeting and not final assembly. So this chapter speaks of the selected set, meaning the final membership of a bundle chosen from the eligible pool, and reserves “admitted to bundle” for what the selected set becomes. Chapter 21 ended on scope-compatible ≠ selected, and this chapter begins there. It does not recompute authority, freshness or scope. It consumes their verdicts. The problem is called assembly here, and the implemented mechanism is Chapter 23’s compiler.
Ranking is not assembly The obvious answer is a ranking: score for relevance, sort, take from the top until the budget is used.
function topK(candidates: Candidate[], budget: number): Candidate[] { const ranked = [...candidates].sort((a, b) => b.relevance - a.relevance); const chosen: Candidate[] = []; let used = 0; for (const c of ranked) { if (used + c.tokens > budget) continue; chosen.push(c); used += c.tokens; } return chosen; } Ranking remains useful for ordering optional candidates within a feasible search. A scalar score assigned independently to each item, however, does not itself enforce hard eligibility, mandatory inclusion, dependency closure, coupled groups, representation alternatives or infeasibility. Nor does it capture bundle-dependent value or cost. A ranking-based algorithm can become an assembler by adding explicit constraint checks and a failure result; the distinction is between a score and the complete decision procedure, not between ranking and engineering.
ranking ≠ assembly ranking ≠ useless retrieval rank ≠ bundle membership Nor is greediness the fault. A greedy method that respects hard constraints, prices dependency closure, handles alternatives and has a deterministic stopping rule may be adequate. The criticism is of naïve per-item greedy ranking.
Hard constraints and soft objectives A hard constraint is a condition whose violation makes a bundle invalid. Examples are the budget, upstream eligibility, mandatory content, dependency closure, coupled-group integrity, preservation obligations and declared exclusions. A soft objective chooses among feasible bundles, for example relevance, coverage, diversity, expected task utility, lower cost or less redundancy. The book prescribes no objective:
hard constraint ≠ large weight constraint violation ≠ poor optimisation Two items in the pool show why. A configuration file from a different project scores 0.99 for relevance. An observation of the backend exists in two versions, a stale compact one of 100 tokens at relevance 0.9 and a current full one of 600 tokens at relevance 0.75. A relevance-only ranking can prefer both mistakes to legal alternatives. The obvious repair is a blended score, relevance minus a stale penalty minus a wrong-scope penalty, and it fails structurally. Unless the penalty is absolute, sufficiently high relevance can buy back illegality, and an absolute penalty is a constraint written awkwardly as a number:
non-negotiable rule → constraint tradeable preference → objective The chapter does not claim that freshness, authority or scope are intrinsically binary. An upstream policy supplies an eligibility verdict for this computation, together with its reason and any uncertainty it chose to tolerate. The allocator enforces that verdict; it must not silently reinterpret an ineligible candidate as a low-scoring option. Consequently, structural validity proves compliance with the supplied verdicts, not the correctness of the upstream judgements. “Legal” is used informally for constraint-satisfying, with no juridical meaning.
Inspect this in the Context Viewer The viewer scene for this chapter works the chapter’s own arithmetic: a 4,600-token mandatory pile against a 4,000 ceiling, with the chapter’s traps annotated — illegal high-relevance items, dependency closure costs, the coupled claim and qualification, four representation alternatives. Open the scene and try to assemble a legal bundle before reading the verdict. What to examine: why ranking fills the wrong pile, and why the correct output here is a structured refusal. Feasibility first; the ceiling is not a target.
Feasibility before optimisation; hard gates before scoring. Constructed illustration from the chapter’s numbers. Open the assembly scene in the Context Viewer
Feasibility before optimisation Before asking which valid bundle is preferable, determine whether any valid bundle exists.
feasibility: does at least one bundle satisfy all hard constraints? if yes → choice: which feasible bundle, under an explicit objective? if no → structured failure feasible ≠ optimal infeasible ≠ poorly ranked no valid bundle ≠ low-quality bundle The opening is a mandatory overflow under its declared accounting convention. If every mandatory obligation has one fixed, independently admissible representation and their dependency closure has exact rendered cost greater than the budget, infeasibility follows. In the general case, however, mandatory obligations can have multiple legal representations, shared dependencies and rendering overhead. The allocator must establish that no permissible combination fits before declaring infeasibility; simply summing the costs of one chosen representation per obligation is not a proof. A structured failure records the violated constraints, the accounting basis, available budget and, where established, a minimum required cost or defensible lower bound. It does not silently truncate or invent a policy relaxation. This assembly failure is distinct from a model safety refusal or Chapter 19’s abstention.
Failure has several causes, and “no bundle” is too vague for diagnosis: mandatory or dependency overflow, no admissible representation, conflicting constraints, exact-render overflow and unresolvable cost or dependency information. The last two require care. A failed estimate can trigger a new allocation attempt rather than prove global infeasibility; an unknown cost or unresolved dependency can instead yield an undetermined result when the allocator cannot certify either feasibility or infeasibility. Unknown is not zero, and an incomplete upstream dependency graph cannot become a guarantee merely because the assembler enforced the edges it received.
Emitting a plausible but invalid bundle is the failure. Correctly reporting infeasibility keeps the contract.
Cost belongs to sets A reference to the earlier incident takes twenty tokens in the proposed fixture. If the model must explicitly call a resolver and that capability is not already exposed, the model-visible resolver definition adds 650 tokens to the relevant bundle. The example assumes both are charged to the same declared input budget. If the runtime resolves the reference without model-visible tools, or the resolver definition is already part of the fixed envelope, that incremental cost is different. The candidate’s local representation cost remains twenty tokens; the additional cost is conditional on the selected set and runtime mechanism:
local candidate cost ≠ incremental admission cost ≠ total closure cost first reference: 20 + 650 = 670 second reference, same resolver: 20 If A needs B and B needs C, selecting A requires closure(A) = {A, B, C}, and the budget prices the closure, not A. When two candidates share a resolver, the marginal cost of a candidate given the selected set is what matters, and a static value-per-token ordering misfires. A candidate that looks like a bargain at 100 tokens but needs a 900-token dependency costs 1,000, and a slightly less valuable one at a flat 500 costs half as much. Chapter 4 argued this from two files, and Chapter 17 priced the commonest dependency, the tool definition.
Dependencies are of several kinds: operational (a model-mediated reference needs a resolver), interpretive (a claim needs its qualification), provenance-related (a claim needs its source marker), policy-related (an unresolved conflict needs both claims and a marker) and representational (an anchor needs sufficient interpreting material). An implication of the form “select X only if Y is selected” can encode many of them, but not every relationship is reducible to a single item-to-item edge. Mutually exclusive alternatives, all-or-none groups and conditions on which representation is chosen need their own explicit constraints or a sound encoding into the same solver. The allocator enforces only relationships supplied and validated upstream; missing edges are not proof of independence.
Cost has several units: candidate-declared estimates, tokenizer-derived estimates, exact cost of the rendered input under a specified tokenizer, and provider-reported usage or billing. These are not interchangeable. Labels, separators, tool envelopes and ordering may alter tokenisation, and some providers do not expose a locally reproducible wire-level count. An estimated fit is therefore provisional. The assembler must either validate against the enforceable input limit using an appropriate counting method and stated margin, select another admissible form, or return a bounded uncertainty/failure result; it must never claim exact feasibility from an unverified estimate or silently truncate the rendering.
The budget it consumes is a declared usable budget (Chapter 4), and the assembler states what that already excludes. Response reserve, a fixed system prompt, tool schemas and wrapper overhead are not subtracted twice. Other runtimes may account for fixed envelope costs upstream: a standing instruction may be a fixed envelope cost and not an ordinary candidate, so the relation is envelope plus mandatory closure plus optional allocation within the usable budget.
Coupled material and alternatives Some material is misleading in part. The claim “migration completed for all tenants” is 120 tokens, and its qualification “except tenant 47” is 80. A bundle with the first and not the second states something false, so:
individually representable ≠ independently admissible A dependency says selecting A requires B. A coupled group says all specified members are selected together or none are. These can sometimes be expressed with implications, but they impose different conditions and deserve distinct audit reasons. For unresolved conflict, Chapter 19 may require both claims and an explicit conflict marker; selecting just the claim with the higher score is invalid. Semantic disagreement does not entail mutual exclusion: two inconsistent propositions may both need to be present as attributed evidence, rather than being merged into one asserted fact.
Representations raise a separate question. The incident report has a full text of 900 tokens, a compact summary of 300, an anchor of 90 and a reference of 20 plus its resolver. These are not one fidelity ladder (Chapters 12 and 18). They are alternative admissible representations that satisfy different preservation obligations at different costs, and a reference is not a lower-fidelity form of the content but a handle plus a recovery dependency (Chapter 13). Two consequences follow:
cheaper representation ≠ admissible substitute more expensive representation ≠ better representation A representation that fails the content’s preservation obligations is not a substitute, however cheap. Declared substitutes should not be selected redundantly, unless policy permits a complementary hybrid, such as an exact value beside an explanatory narrative. Alternative ≠ duplicate universally. Whether two forms are substitutes or complements comes from upstream, since similarity alone cannot say. Two similar candidates may be redundant, corroborating, conflicting or complementary, and the value of a set is not the sum of its members:
value(A and B) ≠ value(A) + value(B) A candidate is an assembly unit with a declared representation and cost. Arbitrary truncation is not part of selection: selection ≠ truncation, and allocation ≠ compression. If nothing fits, an upstream transformation must supply another admissible candidate, or the result is infeasible.
Mandatory material and stopping Mandatory means required for the current computation under the active policy, not globally important. The requirement may be to satisfy an obligation rather than to select a particular text item: a tool capability could have several equivalent legal representations, and task-required evidence may be supplied by one of several independently sufficient candidate sets. Mandatory obligations therefore need not imply mandatory inclusion of every candidate offered to satisfy them. Exact constraints, dependency closure and conflict-preservation requirements remain hard once declared; candidate provenance, including memory origin, confers no automatic priority (Chapter 16).
The budget is a ceiling and not an objective:
used tokens ≤ budget (not used tokens → budget) Unused capacity is not failure, more material is not more value, and less material is not a better bundle. That leaves the mechanism needing an explicit stopping rule other than exhausting the budget. A candidate rule is to stop when no remaining feasible addition has positive expected value under the declared objective, but behavioural value is not known in advance, and the book does not assume the correct rule. How much extra material hurts a given model on a given task is empirical (Chapter 5).
Two more properties concern reproducibility and audit. Determinism requires identical decisions for identical fully specified inputs, including candidate identities, representation versions, eligibility verdicts, costs, relationships, objectives, budget, order-sensitive rendering rules and a stable tie-break. Reproducibility does not guarantee that provider token accounting will match a local estimate. The decision record should say why a candidate was excluded, why another was necessary, which dependency or alternative was selected, why the process stopped, and which constraint prevented assembly. An audit of a solver’s declared inputs cannot independently certify that those inputs were correct:
deterministic ≠ correct deterministic ≠ optimal auditable ≠ correct The last split is structural against behavioural. Budget, closure, mandatory inclusion, exclusivity, eligibility and group integrity can be checked without a model. Helpfulness, distraction and marginal utility cannot. Structurally valid ≠ behaviourally good, and valid ≠ optimal.
The evidence None of this was collected on this book’s fixtures.
Source Type Population/system Verified claim Chapter use Does not establish Qureshi et al., “Budget-Aware Routing for Long Clinical Text” (Findings of ACL 2026) Peer-reviewed Clinical text under a token budget Subset selection under a token budget can be posed as constrained optimisation, and the best method depends on task and budget Selection under a budget is a constrained problem Transfer to coding agents Ghulyani et al., PACMS (arXiv, 2026) Preprint Agent context pools A pooled submodular selection was proposed against recency truncation Neighbouring pooled-selection work Its objective as ours, or validation of this assembler Anthropic context-engineering essay (29 September 2025) Vendor guidance Anthropic’s guidance Curate the smallest set of high-signal tokens Curation, not filling, is the goal That a smaller bundle behaves better Subset-selection evidence is not validation of this assembler. A knapsack captures the budgeted choice and misses eligibility, dependencies, groups, alternatives, obligations and refusal:
knapsack ≠ full assembly model The right model may be two stages, find any constraint-satisfying bundle and then choose among the feasible ones by an explicit objective, and the chapter does not become operations research. Nothing is claimed about the complexity of the general problem, and the practical question is whether a small bounded mechanism produces inspectable, deterministic results.
Ecological gate The corpus of real sessions is empty, so nothing is known about how often these traps arise. Before building a sophisticated allocator, real traces need to show candidate pool, eligible pool and selected set sizes, budget utilisation and unused budget, the frequency of mandatory material and mandatory overflow, dependency and shared-dependency frequency, coupled groups, representation alternatives and redundant admission, unresolved conflict groups, resolver dependencies, exact-render overflow after an estimated fit, cases where top-k differs from constraint-aware assembly and whether that changes behaviour, assembly failures and their reasons, allocator runtime and trace size:
assembly traps are constructible ≠ their production prevalence is known
Proposed experiments Nothing here has been run. Fixtures are small and isolate one mechanism each, and there is no monster fixture, so a failure is diagnosable.
Experiment A: structural correctness. Can an assembler satisfy declared hard constraints, distinguish infeasibility from uncertainty, and return a valid result or a justified failure? No model is involved. Each fixture has frozen candidate representations, dependency and group relations, render rules, a declared tokenizer or cost oracle, and an independently checked feasible-set oracle. In small instances, enumerate possible legal selections to verify whether the claimed infeasibility is genuine; do not treat the allocator’s own verdict as ground truth. Fixtures:
A1, mandatory overflow: budget 4,000 against mandatory closure 4,600 gives infeasible with excess 600 and no bundle; at budget 4,600 it is feasible. A2, shared dependency: two references sharing one resolver. A3, coupled truth: a claim with its qualification. A4, conflict preservation: two claims and a marker. A5, alternatives: several admissible representations with different costs. A6, ineligible high-relevance item: a 0.99 candidate against a legal lower-relevance one. A7, render overflow: estimated costs fit and the exact render does not. A8, no positive optional addition: mandatory material fits and optional material should not fill the space. Baselines: naïve top-k, relevance-per-token greedy, a budget-only knapsack, the constraint-aware mechanism and an independently determined valid-set oracle. The oracle checks feasibility under the full constraint specification; it does not merely provide one convenient bundle. If optimality is tested, the objective and tie-breaking rule are frozen and an exhaustive or independently verified optimum is reported separately. Measures, all mechanically checkable: hard-constraint violations, mandatory-obligation omissions, dependency and group violations, invalid alternative combinations, ineligible admissions, rendered overflow, false infeasibility, unsupported feasibility claims, unresolved-cost outcomes, budget utilisation, repeatability and trace completeness.
Experiment B: behavioural allocation. Where several structurally valid bundles exist, does constraint-aware allocation improve downstream behaviour over simpler selection? Fixtures include distractors competing with evidence, redundant and complementary representations, shared dependencies, coupled claims and qualifications, and conditions where leaving budget unused might help. Candidate sets, task requirements, representations, model, decoding, budget and rendering contract are frozen. First report whether each baseline produced a structurally valid bundle; an invalid top-k result cannot be treated as a clean comparison of two valid allocation objectives. For the behavioural comparison, either use valid-only baselines or separately label violations as their own mechanism. Ordering is held fixed where possible, with unavoidable order differences documented because Chapter 6 predicts behavioural effects. Report performance and resource costs per fixture class, not one pooled winner.
Outcomes that count: validity improves and behaviour does not (its value may be auditability), simple top-k performs equally (delete the complexity), constraints matter only on rare fixtures (the gate decides), dependency-aware cost matters and coupling and alternatives do not (keep what is earned), early stopping hurts, or unused budget goes with equal or better behaviour.
Experiment C, assembly economics, is deferred until A and B leave the question open: whether the runtime and trace cost exceeds what it saves.
What would simplify the book. If a simple constraint-checking greedy allocator matches a more elaborate search on validity, behaviour and cost across representative pools, use the simpler allocator. If naïve top-k behaves equally well but occasionally violates hard constraints, that result does not justify removing the constraints; it may justify a small validation and refusal layer rather than a sophisticated optimiser. If difficult constraint combinations rarely occur in real traces, defer machinery for them without treating their constructed failures as disproven. If extra optional content provides no benefit, the stopping policy need not fill the budget.
What an assembler must do The requirements follow from the failures. Input: an eligible candidate pool, relationships, admissible representation choices, hard constraints, a budget and optional objective signals. Output: a valid selected set or a structured infeasibility result. Along the way it treats upstream ineligibility as exclusion and not as a tradeable score, treats some candidates as non-negotiable, prices a candidate as its closure net of what is selected, keeps coupled pieces together, respects declared substitutes, stops by an explicit rule, hands ordering to Chapter 6’s principles, and validates the exact rendered cost. Ordering can itself change cost through separators and grouping.
The compiler analogy is used only where it explains: explicit inputs, explicit constraints, deterministic validation, structured failure and an inspectable decision record. Assembly does not translate a program, semantic correctness is not statically provable, and the bundle is not thereby optimal.
Handoff Assembly begins after candidate generation and eligibility have narrowed the pool. Ranking can help choose among optional material and cannot express constraints, closure, coupling, alternatives, set-dependent cost or infeasibility. The first question is whether any bundle satisfies the declared constraints. If one does, an allocator may choose among feasible bundles and stop before the budget is full. If none does, the correct output is a structured failure. Structural validity proves only that the bundle obeyed the rules the system declared.
Chapter 23 asks what the smallest deterministic, inspectable mechanism is that can implement enough of this specification to test whether compiling Context is useful.
References Qureshi, K., Martin, G., Peng, Y. “Budget-Aware Routing for Long Clinical Text.” Findings of ACL 2026. https://aclanthology.org/2026.findings-acl.2114/ Ghulyani, M., Singh, A., Bharadwaj, K., et al. “PACMS: Submodular Context Selection as a Pluggable Engine for LLM Agents.” Preprint, arXiv:2606.20047. https://arxiv.org/abs/2606.20047 Anthropic Applied AI team. “Effective context engineering for AI agents.” Engineering essay, 29 September 2025. https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents