Context Is a Bottleneck
Chapter 10 took selection as far as a frozen run has taken it: which memories the present work needs, admitted by an explicit, traceable policy at a fixed budget. This chapter shows selection is insufficient, because admission assumes the admitted memories fit. They increasingly do not — and the Chapter 10 run puts a number on the shortfall, audited for fairness in planning/chapter-14-oracle-audit.md rather than taken on trust. Its ledger oracle reaches perfect required-evidence recall on a mean of 587 estimated tokens; the best real condition spends 1,161 to reach 0.901 recall. Rendered with the source headers and validity marks the reader actually sees, the figures become roughly 805 against 1,505 — the gap survives rendering. The auditable oracle defined in this chapter — content plus the disagreements, licence bearers, and marks a trustworthy system must carry — stands near 696 counted (1,060 rendered) tokens, still roughly 40% below the practical bundle. Roughly half the admitted budget, under a correct frame and over a good candidate pool, goes to evidence the work does not need: the practical bundle’s distractor admission sits near 0.45. That gap is not a selection failure. Selection chose those items for defensible reasons; what is missing is everything that happens between choosing a set and handing a model a context. The system now holds events, beliefs, provenance chains, validity intervals, intentions, derived loops, rules, and prior failures — and the obvious solution, putting all relevant memory into the prompt, fails in ways that are structural rather than incidental. Context is a bottleneck, and what passes through it must be assembled, not merely selected.
Why “put it all in” fails
Take the February compatibility-release task (task-108, as_of: 2025-02-07, for release-121 shipping 2025-02-14) with everything the book’s machinery now considers relevant: the migration decision with its support chain, the backup debt — completed by commit-118, so labelled historical rather than seated as live — the docs scope condition, the fixture warning — likewise historical, completed by commit-120 — the reversibility rule, the corpus_import caller migrations with their trap annotations, the contracted facade that must not be touched. Each item earned its place. Together they exceed the fixed budgets this chapter tests — and the failure modes of stuffing them in are several distinct problems, not one problem called “too long.”
Finite window and cost. The hard limit is obvious: context windows end, and tokens cost money and latency on every task. But the soft limits bite first. Every additional memory raises the price of every future task that carries it, so a policy of admitting everything relevant taxes the system continuously for the possibility of need.
Distraction and dilution. As admitted context grows, the task itself occupies a shrinking fraction of what the model must process. The mechanism here is stated carefully as hypothesis, not fact: beyond some load, task-critical instructions compete with background memories for influence over the output, and the spec’s distraction cost exists to measure the effect rather than assume it. No broad claims about attention mechanisms appear here; what needs measurement is behavioural — does constraint adherence degrade as admitted context grows at fixed relevance?
Duplication. The decision record, the runbook echo, and the August restatement all carry the same belief in different words. Admitting all three spends budget triplicating one fact while crowding out the fixture warning. Deduplication is not retrieval’s job — each copy is genuinely similar to the query — it is an assembly operation over the admitted set.
Contradiction in context. Historical background and current belief disagree by design: the March SQLite passages sit beside the July migration. Admitted raw, without their temporal resolution attached, they present the model with a contradiction the system already resolved and ask it to resolve it again, under task pressure, from wording. The Chapter 8 resolution must travel with the memory, not stay behind in the store.
Staleness smuggled in. A memory admitted for background — the spent 2023 discussion — carries claims that were true then and false now. Without validity marking, background becomes misinformation. The assembly must label what selection admitted: current, historical, superseded, uncertain.
Lost task focus. The subtlest failure: a context dominated by history reframes the task as summarization. The model answers what happened fluently and never performs what is needed. The task statement drowns in its own background.
Each failure earns a different operation, which is why the chapter separates them instead of prescribing “compression” as a single fix.
Retrieval is not assembly
The separation this chapter establishes, matching the spec’s failure-attribution categories:
Retrieval asks: what candidates might matter?
Context assembly asks: what exact representation of memory should this model receive for this task?
Selection ends with a ranked set. Assembly turns that set into a prompt: which items survive, in what order, grouped how, deduplicated against what, compressed to what degree, cited to what depth, with uncertainty marked where. Two systems with identical selection and different assembly can behave differently on the same task — the one-level-up analogue of Chapter 13’s same-store experiment — which makes assembly a layer with its own measurements rather than a formatting detail.
Possible operations, listed as emerging requirements rather than developed solutions:
- select — the policy output, Chapters 12–13;
- order — task-critical first, background last, traps flagged rather than buried;
- group — the support chain travels with its belief; the loop travels with its triple;
- deduplicate — echo documents collapse to one representative with a count, not three admissions;
- compress — shorten with provenance preserved, a later block’s problem, not this chapter’s;
- cite — every assembled claim traceable to store and artifact, the Chapter 7 discipline at the point of use;
- expose uncertainty — calibrated marks where state is genuinely unresolved;
- preserve provenance — chains and triples survive assembly rather than dissolving into prose.
The list stops short deliberately. Consolidation, durable compression and forgetting now meet as one long-term growth problem in Chapter 15; outcome adaptation and reusable procedures meet at the memory-learning boundary in Chapter 16. Building them now would steal the later chapters’ experiments — the same restraint Chapters 1–8 showed toward task state.
Conceptually, and illustratively rather than as tested code, the accumulated pipeline now reads:
def assemble_context(task, memories, budget):
# Illustrative: policy-selected memories in, budgeted
# assembled context out. Every operation below is a
# placeholder for the next block's experiments.
ranked = retrieve_for_task(task, budget) # Ch 10, 12–13
grouped = group_with_evidence(ranked) # Ch 7/11 triples ride along
deduped = collapse_echoes(grouped) # Ch 7 derivation edges
marked = attach_validity(deduped) # Ch 8 intervals
return fit_to_budget(marked, budget) # Chapter 15 asks what survives long-term
The experiment
Selection is frozen at Chapter 10’s C5 admitted set — eleven tasks, the same bundles the frozen run recorded. C5 is the input rather than C6 deliberately: C6 adds coalescing and precision falls from 0.474 to 0.464 at equal recall, so C6 belongs in the comparison, not the foundation. Conditions vary only what happens after admission: raw render (A0), reordering (A1), extractive dedup (A2), grouping (A3), validity marking (A4), budget dropping with a reason per drop (A5), all composed (A6), the ledger-content oracle (CO-content), the auditable oracle with disagreements, licence bearers, and marks restored (CO-auditable), frozen C6, empty, and random-drop controls. Budgets sweep 384 through 1536 estimated tokens plus unconstrained. The frozen run is experiments/benchmark/runs/ch14-20260920T174259Z-context-assembly/, from solution/context_frames/assembly.py (assembly-v1, drop policy drop-preference-outside-first-v1), zero model calls for the evidence suite plus a bounded 55-call reader probe through the frozen Chapter 10 reader (llama3.1:8b) and key-claim scorer.
At unconstrained budget, over the eleven tasks:
condition required-recall counted rendered contra licence
A0 raw 0.902 1174 1505 1.00 1.00
A2 dedup 0.902 1156 1480 1.00 1.00
A6 composed 0.902 1156 1747 1.00 1.00
CO-content 1.000 587 805 0.00 0.40
CO-auditable 1.000 696 1060 1.00 1.00
C6 (Ch10) 0.902 1161 1644 1.00 1.00
Three facts stand out. First, dedup saves eighteen counted tokens: genuine echo redundancy is nearly absent from C5-admitted sets, because selection already declines to admit most of it. The bottleneck is not repetition. Second, the content oracle drops contradiction entirely (0.00) and most licences (0.40): the ledger-minimum set is not a fair ceiling until the auditable oracle restores them for 109 tokens. Third, marks are not free in the render the model sees: A6’s rendered cost (1,747) exceeds raw (1,505) even as counted cost falls — audit metadata and reader-visible tokens are different budgets, and the trace keeps the full lineage either way.
The budget sweep is where assembly earns or loses. Required recall against budget:
budget A6-composed random-drop CO-auditable
384 0.58 0.58 1.00
512 0.58 0.61 1.00
768 0.72 0.67 1.00
1024 0.82 0.79 1.00
1280 0.85 0.89 1.00
1536 0.90 0.90 1.00
The composed policy beats random dropping on what random cannot see: at 768, contradiction preservation is 1.00 against 0.57 and licence preservation 1.00 against 0.38. But the guarantee costs recall under pressure — at 512 random dropping keeps more required evidence (0.61 vs 0.58), because the policy spends scarce budget protecting disagreement and licences that the ledger does not score as required. That trade is visible rather than hidden: every drop carries its tier, saving, and cover, as in this T1-architecture case at 768, where the required mb-arch-spine falls with covered_by=none while mb-ch09-result falls covered by the dissent it shares a group with — and the dissent itself survives.
The reader probe sharpens the picture without settling Chapter 12’s question. Mean key-claim coverage through the frozen reader, forbidden-claim rate zero everywhere:
A0 full (1174): 0.879
A6 at 768 (659): 0.955
A6 full (1156): 0.894
CO-auditable (696): 0.879
C6 (1161): 0.939
A smaller assembled context (659 tokens) answers better than the full raw context (1,174) on the existing scorer — the same 0.955 the frozen Chapter 10 C6 reached — while required-evidence recall at that budget is only 0.72. Evidence sufficiency and reader sufficiency disagree here in the useful direction, and in a way that belongs to Chapter 12: the reader answers from partial evidence, and one task’s coverage moves with ordering alone. The headline of this chapter stays bounded to assembly, evidence preservation, and cost.
Policy learning follows the Chapter 10 and 11 discipline. The tempting repair — treat contradiction as ordinary budget weight when a long dissent crowds out supporting evidence — is proposed as an immutable drop-policy version, replayed over the suite at budget 512, and rejected: it saves 9.2 tokens and even nudges recall, but contradiction preservation collapses from 1.00 to 0.14. A cheaper context bought with lost disagreement stays out.
Book result. Assembly earns a distinct layer with a Type B profile and a Type C headline. Simple extractive dedup captures almost nothing (18 of 1,174 tokens), so the bottleneck is not repetition but admitted distractors — yet under a hard budget the composed policy preserves contradiction and licences where random dropping does not, and at 768 tokens answers better than the full raw context on the existing scorer. The fair oracle gap is real but narrower than Chapter 10 suggested: 696 auditable tokens against 1,156 assembled (1,174 selected), with required recall 1.00 against 0.90. Demotion clauses: controlled project fixtures, one primary reader, estimated token counts, C5-admitted evidence only, no generative compression (Chapter 16’s problem, deliberately unbuilt here).
Where the middle of the book lands
Fourteen chapters now form one progression, each step forced by the last:
retrieval
→ reconstruction
→ belief
→ unfinished intention
→ situational relevance
→ active context
The system retrieves history, reconstructs what happened, justifies its beliefs, tracks them through time, maintains what remains undone — including consequences nobody stated — selects what the present task needs by explicit policy, and assembles it for a bottlenecked context. What it cannot yet do is the other half of the bottleneck’s demand: it can only discard or shorten by fiat. It has no principled way to preserve what matters across tasks in less space, to let outcomes strengthen or weaken what is kept, to shed what no longer serves, or to turn repeated success into reusable procedure.
A system can now remember too much. The next problem is learning what to preserve, what to compress, and eventually what to forget.
Three boundaries hold for what comes next. This chapter shows a smaller assembled context preserves the evidence and the existing reader’s answer-level behaviour; it cannot yet establish that better context improves richer downstream project behaviour — constraint adherence, failed-approach avoidance, open-work continuation — and Chapter 12 owns that instrument. Wrong-frame behaviour under a tight budget is left to Chapter 13; the headline experiment uses declared, correct WorkFrames throughout. And nothing here rewrites the durable store: every compact representation is derived, ephemeral, and rebuildable, and long-term storage compression belongs to Chapter 15.
What long-context research adds to assembly
LongBench, RULER, and InfiniteBench measure long-context behaviour across tasks and lengths rather than inferring it from an advertised window. Lost in the Middle makes position an experimental variable: the same evidence can have different effects depending on where it appears.
An assembler should therefore record inclusion, position, representation level, token cost, and attached validity/provenance marks. E-14 can then distinguish exclusion, dilution, placement, and representation failures.
The papers show that long inputs can remain difficult; they do not prove that grouping, deduplication, or validity marking repairs them. Each operation needs an isolated matched-budget condition. The useful output is a performance-versus-load curve for every variant, with the breakpoint where an operation begins to help. This prevents context assembly from becoming unconditional ceremony.
Research foundations
Long-context benchmarks consistently show that a larger context window does not make context construction irrelevant. LongBench measures multi-task long-context performance; Lost in the Middle exposes position sensitivity; RULER adds controlled tests of retrieval and aggregation at increasing lengths; InfiniteBench stresses models over extremely long inputs. This evidence supports treating context as a limited behavioural resource, although the chapter’s particular assembly operations still require their own matched-budget experiment.
Two recent results sharpen the design without supplying the mechanism. SARA (Jin et al., ACL 2026) optimises RAG under fixed token budgets by pairing a small set of text passages with compressed semantic vectors, and reports answer gains from the hybrid. The question it leaves this book is whether structured project memory — support groups, validity intervals, triple licences — already supplies the coverage information SARA buys with vectors; this chapter keeps every representation extractive and inspectable rather than finding out the other way. Adaptive-k (Taguchi et al., EMNLP 2025) shows fixed passage counts waste tokens or omit evidence and selects a query-specific count by a single-pass score threshold. That is adaptive sizing, the complement of this chapter’s fixed-budget fitting; the sweep above measures quality against budget first, and adaptive budgets stay reserved until a run earns them.
References
- LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding (2023).
- Lost in the Middle: How Language Models Use Long Contexts (2023).
- RULER: What’s the Real Context Size of Your Long-Context Language Models? (2024).
- InfiniteBench: Extending Long Context Evaluation Beyond 100K Tokens (2024).
- SARA: Selective and Adaptive Retrieval-augmented Generation with Context Compression, ACL 2026.
- Efficient Context Selection for Long-Context QA: No Tuning, No Iteration, Just Adaptive-k, EMNLP 2025.
