# Bring Back Only What You Need
# Book: Context From First Principles
# Chapter URL: https://programmer.ie/books/context/14-chapter/
# Prose fingerprint (sha256, normalized): d6936244771ef853c400acdb2819b34b2099304796564cdc32b7fff39ca142fd
# Snapshot generated at build time by Hugo. This is the complete rendered prose
# of the chapter; it is not a transcript of the source file.

Chapter 13 left the model holding a tidy live context and a shelf of external artifacts:
Incident 17 — lock-order inversion. Evidence: artifact://incident-17 plus five more references of the same shape. Then the task changes: the user asks why the deadlock appeared only under concurrent import. The incident artifact probably matters, and so might the benchmark report, the migration plan and the compiler run. Loading everything recreates within one turn the occupancy problem Chapter 13 solved. Leaving was only half the problem.
This chapter starts after Chapter 13’s infrastructure exists: an external representation, a valid recovery contract, and a runtime that can resolve it when asked. Its question is:
Given a larger information world than can fit in the current context, which available information should become model-visible for this computation, in what representation and extent, and when should further disclosure occur?
Candidates are not context A candidate is an identified opportunity for admission: a content object or representation, plus the metadata an admission policy needs to reason about its eligibility, expected cost, provenance, dependencies and available forms. It is not necessarily a document, and it need not have been read. It can be a reference, metadata about a representation, an already materialised representation, or a fragment. Four identities stay separate:
candidate identity, content identity, representation identity, fragment identity Fragment identity is not artifact identity: five sections of one artifact are five distinguishable fragments of one source, not five independent sources. Their contents may overlap, and counting each fragment as a separate source can overstate corroboration or provenance diversity. Chapter 27 builds candidates, and how they are found is not this book’s subject. Exact lookup, text search, embedding retrieval, database queries, tool search, agent navigation and memory recall are candidate-generation mechanisms, and all end at the same boundary, a candidate pool. The chapter takes the pool as input and does not teach or compare them. A pool entry must retain enough source and fragment identity to prevent repeated excerpts from masquerading as independent evidence.
candidate generated → admission requested → recovered → admitted to bundle → behaviourally implicated → helpful Each step is a different claim, and the shorthand the chapter protects is:
retrieved ≠ admitted ≠ used ≠ helpful
In the shorthand, retrieved means surfaced to the runtime as a candidate; it does not mean that the artifact payload has been recovered or observed by the model. Admitted means material was placed in the assembled bundle at the observation boundary, not that the provider’s final rendering is known. Used means behaviourally implicated under a stated operational test, not proof of an internal attention path. A search result carrying an identifier and title is not the artifact payload:
candidate generated ≠ artifact recovered candidate ≠ context Chapter 13’s epistemic boundary carries over. A policy may use resident metadata, such as “lock-order inversion, artifact://incident-17”, to decide whether to resolve the artifact. It may not attribute to the model details that exist only behind an unresolved reference:
reference present ≠ artifact observed
Where the decisions live Five stages have different owners, though an implementation may combine them in one component. The boundaries are logical and should remain identifiable in the trace:
Candidate builder (Chapter 27) produces normalised candidates. Admission policy (this chapter) proposes candidate representations and extents using information it is permitted to inspect; a decision made from metadata need not load the payload. Recovery resolves the selected external representation under Chapter 13’s contract and checks identity, version and access before it can be admitted. Assembler (Chapter 23) reconciles recovered, eligible material with budget, order, authority, scope and freshness requirements and emits the bundle. A policy request is not proof of final admission: budget feasible ≠ admission justified, and admission proposed ≠ admission applied. Runtime disclosure controller may initiate another candidate, recovery and admission cycle if the available evidence does not warrant stopping, and asks the assembler for a new bundle. Some systems recover a preview before choosing an extent; others decide from metadata alone. The trace must say which information was available at each decision, rather than imposing one fixed execution order on every implementation.
Candidate builder (Chapter 27) produces normalised candidates. Recovery resolves an external representation under Chapter 13’s contract. Admission policy (this chapter) requests or selects candidate representations and extents for a computation. Assembler (Chapter 23) reconciles the eligible candidates with budget, order, authority, scope and freshness requirements and emits the bundle. Admission is not ordering, is not final assembly, and is not budgeting: budget feasible ≠ admission justified. Runtime disclosure controller may invoke another recovery and admission cycle when current material seems insufficient, and calls the assembler again with a different pool. Authority, freshness and scope remain metadata in this chapter. Retrieval rank is not authority. The chapter preserves provenance, version and scope metadata for Chapters 19 to 21 and adjudicates nothing.
Inspect this in the Context Viewer The viewer scene for this chapter runs the funnel on ten available artifacts: seven surfaced, two admitted, with a candidate miss, a recovery failure and a policy rejection at three separate stages. Open the scene and locate each failure precisely. What to examine: which needed item never entered the pool, which stalled at recovery, and which the policy rejected — and what each location implies for the remedy. Absence from the bundle is never read as irrelevance here.
Candidate miss, recovery failure and admission failure scored at separate stages. Constructed illustration. Open the funnel scene in the Context Viewer
What can fail The failures differ, and scoring one as another misdirects the fix. Several may coexist in the same trajectory, so they are diagnostic stages rather than mutually exclusive explanations:
Candidate-generation failure: required information never entered the pool. Admission cannot repair it.
Recovery-integrity failure: a requested candidate’s contract did not resolve to the intended representation (Chapter 13).
Initial admission failure: the required candidate was in the pool but the policy did not select it, or a selected candidate was not actually admitted. The trace should distinguish policy rejection from assembler rejection.
Representation or extent mismatch: the right source was chosen but the admitted view lacked a required field or relation.
Premature disclosure stop: the runtime stopped under a fixture criterion that required further evidence available through a permitted disclosure.
Over-disclosure: it continued expanding after the fixture’s specified sufficiency condition was already met; this is a resource judgement, not necessarily a behavioural failure.
Downstream use failure: required material was present at the observation boundary but the tested behaviour did not use it correctly (Chapter 5).
Candidate-generation failure: required information never entered the pool. Admission cannot repair it.
Recovery-integrity failure: a selected candidate’s contract did not resolve to the intended representation (Chapter 13).
Initial admission failure: the required candidate was in the pool and was rejected or never requested.
Representation or extent mismatch: the right candidate was chosen and the material admitted lacked a required field or relation.
Premature disclosure stop: the runtime stopped while hidden material was still needed.
Over-disclosure: it kept disclosing after the current material sufficed.
Downstream use failure: the material was present and not used correctly (Chapter 5).
candidate miss ≠ recovery failure ≠ admission failure
The cost of admitting everything Suppose ten artifacts contain information the task needs in three. A generator returns seven: the three required and four irrelevant. Its candidate recall was perfect under these fixture labels, and admitting all seven can still fail through bloat, interference, position and cost. Retrieval recall is not admission precision. Candidate-level precision also understates occupancy: one useful 10-token artifact beside one irrelevant 20,000-token artifact yields 50 per cent candidate precision, yet almost all admitted payload tokens belong to the irrelevant artifact. That arithmetic does not establish behavioural harm or prove those tokens are safe to remove. The chapter therefore reports token-weighted occupancy alongside candidate-level results, not instead of them. Required, supporting, oracle-unnecessary and distractor tokens, plus reference and metadata tokens, are separated only where fixture labels support that attribution; mixed-purpose spans are not forced into mutually exclusive token labels. Source-level counts (unique artifacts, fragments and tokens per source) are reported as well.
Irrelevant material can harm, on some evidence. Amiraz and colleagues define a passage’s distracting effect as the probability that a model fails to abstain when given a passage that cannot answer the question. On seven models from 3 to 70 billion parameters they report strong correlation across models, and on Natural Questions, adding a hard distracting passage to the gold passage cut accuracy by 6 to 11 points depending on the model. They also report that higher-ranked irrelevant results have greater distracting potential and that adding a reranker increases the average distracting effect. The population is question answering over retrieved passages, not agentic artifact admission. It shows that retrieved irrelevant passages can degrade generation, and that under that retrieval setup ranking can concentrate some hard distractors near the top. A fixed top-k should therefore not be assumed behaviourally neutral. It is still a simple, serious baseline, and a result in which it matches a more elaborate policy would be valuable.
Relevance, utility and use Similarity ends at the candidate boundary. Retrieval score is not admission utility, and relevant is not helpful:
relevant ≠ helpful
Utility is marginal and depends on the bundle. What a candidate adds depends on what is already resident, so ΔB(x | C) denotes the behavioural difference between otherwise controlled conditions with and without x, not a value the candidate carries intrinsically. The comparison must specify the task, model, decoding, measurement and how added material changes length and position. A difference in outcome after admission may reflect those changes as well as the candidate’s content:
candidate utility ≠ candidate property
Admission utility is task-relative and multidimensional: the behavioural effect of adding a representation to the current bundle, considered alongside the resources and possible interference associated with the addition. Task success, tokens, latency, cost and interference stay separate unless a deployment policy defines a scalar. A candidate is behaviourally helpful only relative to a comparison of B(C + x) and B(C) under declared acceptance criteria and controls; a single successful answer is not a marginal-utility measurement. Redundant is not irrelevant either. A second source restating a required fact may corroborate it, resolve an ambiguity or add independent provenance; repeated fragments from one source do not automatically provide that independence. The fixture declares when duplicate evidence is unnecessary under frozen obligations. Labels are candidate-by-task-by-baseline judgements, not permanent properties of an artifact.
Used has to be an observable construct. Whether admitted material was used could mean the model referenced it, its behaviour changed when it was removed, or a counterfactual test came out a particular way, and these are not equivalent. The chapter makes no claim about mechanistic use:
model mentions x ≠ x caused the behaviour
Representation and extent Two choices are easy to conflate. Representation choice asks which legal representation of the candidate is admitted (Chapters 11 and 12). Extent asks how much of that represented content comes in:
representation ≠ extent smaller extent ≠ fewer satisfied obligations Admitting a 300-token exact section of a 30,000-token artifact is extent selection within that source representation. Generating a new 300-token summary of the whole is transformation (Chapter 11). Admitting a pre-generated structured record is representation selection (Chapter 12). A ten-token exact identifier can satisfy an identifier obligation perfectly, so extent is judged against preservation obligations and not against a scalar of detail. Selecting exact spans may still remove the qualifiers or relations needed to interpret them; the selected extent must be evaluated as a whole, including any required surrounding context. If the current policy requires an exact field or relation, only views independently shown to satisfy that obligation are eligible. A task-specific generated extraction confounds extent selection with transformation and must be recorded as such.
An expansion can do three different things, and a trace has to say which: choose another already-valid representation of the same content object, bring more of the same source representation (extent), or add another candidate. “Number of expansions” is uninterpretable without that.
Progressive disclosure and the stopping problem Progressive disclosure delays materialising or admitting richer or more extensive information until the current computation gives evidence that more is needed.
It is not progressive summarisation, and it is not a representation hierarchy. Real artifacts have no common ladder: a structured record may serve better than a section, and two exact spans may beat the whole. A disclosure path from metadata to a semantic cue, a section and the full artifact is one implementation pattern, not the canonical one. Chapter 12 moves a resident object through representations, and here material is drawn in from non-resident candidates.
At a disclosure boundary, the runtime faces a sufficiency decision: is the evidence currently admitted adequate for the task and declared acceptance criteria, or is further disclosure warranted? The policy may return sufficient, insufficient or uncertain and must specify how it handles uncertainty and exhausted budgets. This is not necessarily a separate decision for every undisclosed candidate x; candidate choice follows only if the controller decides to continue. A model reporting that it is confident is not evidence of sufficiency:
model confidence ≠ context sufficiency
Sufficiency is relative to the task, its acceptance criteria and its obligations, and the criteria must be defined independently of the policy being evaluated. There is a structural problem: the policy can act only on the admitted evidence, candidate metadata, available previews and other permitted state. The fact that would expose an important omission may sit behind the next disclosure. A preview can help predict sufficiency but cannot certify it merely by appearing coherent; a stopping claim remains provisional unless a task-specific check establishes it:
How can a system decide it has seen enough when the information proving it has not may be behind the next disclosure?
Two errors are the price. Stopping too early leaves required information unobserved, as when an incident cue records a fix while the detail section confines it to the import path and the export path still deadlocks. Continuing too long adds cost, bloat and interference without benefit, and a deterministic policy can over-expand as easily as a model can wander:
premature stop ≠ over-expansion
Eager and just-in-time admission trade differently. Eager preloading buys fewer round trips and immediate evidence, at the price of over-admission. Just-in-time starts lean at the price of steps, latency and misses. Anthropic’s context-engineering guidance describes a hybrid in which some data is retrieved up front for speed with further exploration at the agent’s discretion, and warns that without guidance an agent can chase dead ends or miss key information. As a policy hypothesis, preload material with high predicted near-term need and disclose uncertain material on demand. Need is a prediction, and hindsight belongs to the oracle. Deterministic is not correct, and the same holds for a deterministic gate. Whether the model or the runtime drives disclosure is an architecture choice, with a deterministic rule, a learned policy, a model request, an evaluator trigger and hybrids as options, and the chapter crowns none.
The evidence Source Evidence type Verified claim Chapter uses Does not establish Anthropic, “Effective context engineering for AI agents” Vendor guidance Embedding-based retrieval before inference versus runtime loading through lightweight identifiers; progressive disclosure described as incremental discovery through exploration; a hybrid of some data up front with further exploration; risk of dead ends and missing key information without guidance The pattern and its stated risk That it improves behaviour Anthropic, “Code execution with MCP” (Jones and Kelly, November 2025) Vendor engineering article Tools presented on a filesystem and read on demand; a search tool with a detail-level parameter; results filtered in code before reaching the model Progressive disclosure and pre-context filtering as an implementation pattern Behavioural benefit Amiraz et al., “The Distracting Effect” Peer-reviewed paper (ACL 2025 proceedings) As above: seven models, 6–11 points on Natural Questions with hard distractors, higher-ranked irrelevant results more distracting, reranking raising the average effect That retrieved irrelevant passages can degrade generation Agentic artifact admission; any general rule about rankers Proposed experiments Nothing here has been run. The corpus of real sessions is empty, so how often ordinary work presents ambiguous, multi-candidate admission is unmeasured.
Ecological gate. Before elaborate machinery, describe real traces: candidate pool sizes and payload sizes, how often more than one candidate is available, redundancy, the prevalence of required candidates, later expansions, the share of candidates actually admitted, the latency of disclosure, and how common large distractors are.
an admission policy can help ≠ real workloads present enough ambiguity to justify it
Experiment A: static admission. Given the same frozen candidate pool and resident baseline, what should enter the next bundle? Frozen: task, resident context, pool, metadata visible to each policy, candidate identities and versions, ranking, model, decoding, budget and independently defined fixture obligations. Conditions: no external admission; preload all where feasible; fixed top-k chosen in advance; a deterministic metadata-only policy; a model-assisted policy if its decision cost is included; and an oracle. The oracle identifies sufficient candidate sets relative to frozen obligations and may identify minimal sets by subset inclusion or a separately stated resource measure. It does not claim that a set guaranteeing required information will guarantee model success. Oracle labels are fixed before a run and not derived from its model outputs. Several different sets may satisfy the same obligations, so simple item precision and recall can penalise valid substitutes. Primary measures are obligation coverage, missing obligations, admitted and unnecessary tokens, task outcomes and budget violations; candidate precision and recall are secondary where unique item labels are justified. Report selection requests separately from successfully recovered and assembled admissions, and include policy overhead: model tokens, recovery calls, latency and evaluator calls.
Representation and extent sub-experiment. For a source known to contain required evidence, compare its exact full representation, an exact selected section or span, and a pre-generated compact or structured representation independently checked against the same applicable obligations. Nothing is generated during the condition. Matching the underlying source fixes candidate identity, but an exact span and a transformed representation change different factors. Treat them as distinct contrasts: extent-only comparisons hold representation type fixed; representation comparisons hold the required obligations and evaluation target fixed while reporting achieved size. Measure obligation survival, active tokens, behaviour, missing facts and, for transformed representations, unsupported additions. This examines what form and how much to admit without conflating those choices with which source to select.
Experiment B: progressive disclosure. When should the runtime disclose more? Frozen: candidate universe, legal disclosure options and their costs, task and obligations, initial resident view, budgets, decisive hidden facts and independently labelled sufficiency states after each permitted step. Conditions: eager full disclosure where feasible, fixed schedule, deterministic sufficiency policy, model-requested disclosure, hybrid, and oracle stopping, simplified if implementation cost is prohibitive. Six adversarial fixtures carry the traps: a sufficient-looking preview hiding a reversing qualification; a large plausible distractor; a small decisive low-ranked item; redundant corroboration; a required source whose selected view lacks an exact field; and an initial context already sufficient. A wrong stop is a stop at a state that fails the frozen sufficiency criterion while an affordable permitted disclosure could satisfy it; report separately cases where the required evidence is unavailable or the budget prevents expansion. An oracle-unnecessary continuation discloses more after the current state meets that criterion; it is a fixture-relative resource judgement, not a claim that additional evidence has no possible robustness value. Report both types, rounds, tool calls, cumulative latency and cost, and whether every expansion adds another candidate, increases extent or switches representation. Repeated steps within one fixture are not independent tasks.
Behavioural outcomes include task success, exact-constraint preservation, unsupported claims, wrong conclusions and abstention, and correct output is not read as use. Where provider semantics and telemetry support it, first-divergence and reuse effects are recorded, and mutation radius stays a proposed concept.
Outcomes that count. Preload-all matches the sophisticated policy under realistic budgets, or top-k matches an elaborate gate. A gate saves tokens while behaviour worsens, or improves behaviour at higher cost, which is a trade-off and not dominance. Irrelevant candidates cause no measurable harm on the tested models. Disclosure stops too early often, or wanders enough that overhead erases the benefit. Representation and extent matter more than candidate selection. Multiple equivalent sufficient sets dominate the evaluation. Real candidate pools turn out tiny and unambiguous.
If preload-all or a frozen top-k matches more elaborate policies on behaviour at comparable realised resource cost across the tested fixtures and models, the evidence does not justify the more elaborate gate in those conditions. Likewise, an improvement on fixtures constructed specifically to require staged disclosure does not establish that those opportunities occur often in real workloads. The distinctions survive either result: candidate ≠ context, candidate miss ≠ admission miss, retrieved ≠ admitted ≠ helpful, representation ≠ extent, and reference present ≠ artifact observed.
What the agent creates next This chapter assumed a universe of candidates that the agent moves through. Agents also manufacture context. Plans, hypotheses, tool observations, summaries and intermediate artifacts continuously create new future candidates, including misleading ones whose retrieval can steer what follows. Retrieval generates possibilities and admission creates context, and what happens when the agent starts generating the possibilities itself is the question Chapter 15 inherits.
References Anthropic Applied AI team (Rajasekaran, Dixon, Ryan, Hadfield, et al.). “Effective context engineering for AI agents.” Vendor engineering guidance, September 2025. https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents Jones, A., Kelly, C. “Code execution with MCP: Building more efficient agents.” Anthropic engineering article, 4 November 2025. https://www.anthropic.com/engineering/code-execution-with-mcp Amiraz, C., Cuconasu, F., Filice, S., Karnin, Z. “The Distracting Effect: Understanding Irrelevant Passages in RAG.” arXiv:2505.06914, May 2025; ACL 2025 long papers. Results as reported in the full text (Table 1, Figures 2 and 4). https://arxiv.org/abs/2505.06914