# More Is Not Better
# Book: Context From First Principles
# Chapter URL: https://programmer.ie/books/context/05-chapter/
# Prose fingerprint (sha256, normalized): acdce331feb8d805f0b993fcce0aa12b9ae0b402d6528043d7f87f431fea581b
# Snapshot generated at build time by Hugo. This is the complete rendered prose
# of the chapter; it is not a transcript of the source file.

The seductive assumption runs like this. Useful information is good. The model has spare capacity. Therefore adding more potentially useful information should be better, or at worst harmless. It is easy to build a system on that reasoning: the whole repository goes into the window just in case, every retrieved document is kept because retrieval must have returned it for a reason, history is never truncated because something old might matter. The budget chapter showed that admission costs capacity and money. This chapter asks whether it can cost behaviour.
The proposition is narrow. Added context can reduce measured task performance even while the request stays within hard capacity. It is not the claim that more context is bad, that there is a length past which context always harms, or that one mechanism explains the failures. Capacity and behavioural usefulness are different properties, and presence in context does not guarantee successful use. This chapter reviews controlled evidence for that, marks where the evidence stops, and states what the book will test itself.
Four statuses, not one “The model handles the context” conflates four statuses that must be separated. They are operational statuses under a specified evaluation, not properties of information. They also are not guaranteed stages of one process: a model may answer correctly without being able to identify the supporting passage when subsequently asked, and a direct retrieval probe can change the task being measured.
Present. The required information is in the assembled context at the observation boundary of Chapter 2. This is a property of the bundle, checked by inspection.
Locatable. Under the tested condition, the model can recover or identify the relevant information. This is a behavioural measurement.
Usable. Under the tested condition, the model correctly applies the information to the required inference or action. This is also behavioural.
Helpful. Including the information, or the context condition that contains it, improves a chosen end-to-end outcome relative to an appropriate comparison. This is a comparative outcome.
These statuses distinguish questions an evaluator can ask, but they do not form a logically necessary progression. A successful end-to-end answer need not entail success on a separate explicit-location probe, and a successful location probe does not demonstrate correct application. The statuses also give a diagnostic protocol, not a causal classifier. Given a failure, inspect the capture for presence first, remembering that presence is established at the host observation boundary rather than at the provider. If it is present there, asking for the fact directly tests locatability under a new condition; success or failure cannot by itself diagnose the original attempt. Supplying the fact in isolation probes application with a changed context; success in isolation and failure in the full bundle are consistent with interference but do not isolate its cause. Comparing a bundle with and without the fact probes helpfulness under that comparison, but removal can also change token count, rendered position and coherence. Each probe narrows possible explanations when interpreted alongside controls; none identifies a mechanism on its own.
flowchart TD F["A failure on the task"] --> A("Present?<br/>Inspect the bundle.") A -->|no| A0["Absent evidence"] A -->|yes| B("Locatable?<br/>Does the model find it<br/>when asked directly?") B -->|no| B0["Consistent with a<br/>locatability failure"] B -->|yes| C("Usable?<br/>Can it apply the fact<br/>when the fact is isolated?") C -->|no| C0["Consistent with a<br/>usability failure"] C -->|yes| D("Helpful?<br/>Does the full bundle beat<br/>the bundle without the fact?") D -->|no| D0["No demonstrated helpfulness<br/>under this comparison"] D -->|yes| D1["Helpful under this comparison"] Different phenomena, not one “More context” bundles several things that need not share a mechanism. Overflow is a hard capacity failure. Bloat is not established merely by the bundle being large: under this book’s operational definition, material is bloat only when its removal or replacement preserves required behaviour while reducing resource use under a stated comparison. Interference is a deterioration associated with added or changed surrounding material under a controlled comparison. Position sensitivity is a behavioural difference when the same relevant content moves. Length-associated degradation is a measured decline across input lengths under defined evaluation conditions. A longer input is not necessarily more distractors, a different position or a harder task, and the literature warns against assigning every decline a single explanation.
What Lost in the Middle showed The study the field reaches for first is Liu et al.’s “Lost in the Middle,” and its fame exceeds its scope. The authors tested language models of that period on multi-document question answering and key-value retrieval, moving the relevant information within the input. Performance was often highest when the relevant information was at the beginning or end and dropped substantially when it was in the middle, including for models built for long contexts. The paper was published in a journal in 2024, after a 2023 preprint.
The population was QA and retrieval, and the models were of that period. It did not study agents with tools, and the pattern was a U-shape in those tasks and not a law of all models at all lengths. What survives those qualifications is a dissociation: under those conditions the information was present in every condition and behaviour differed anyway. Rendered position can be behaviourally relevant. That is all this chapter needs before Chapter 6 takes up order.
Beyond the needle: RULER Hsieh et al.’s RULER (COLM 2024) extends the needle test with varied needle types and quantities and with multi-hop tracing and aggregation tasks. It uses thirteen tasks on seventeen long-context models, with length and complexity configurable. The authors report that despite near-perfect accuracy on the vanilla needle test, almost all models showed large drops as length and complexity grew, and that although every model claimed a context of 32K tokens or more, only half maintained satisfactory performance at 32K. “Satisfactory” is the authors’ criterion, and the population is the 2024 generation. The lesson is that success on a simple retrieval test did not carry over to harder uses in that study.
Application tasks diverge: HELMET Yen et al.’s HELMET (ICLR 2025) evaluates fifty-nine long-context models on seven application-centric categories with lengths up to 128K tokens. Its central findings are that synthetic tasks such as needle retrieval do not reliably predict downstream performance, and that the categories show distinct trends with low correlation between them. Long-context ability is therefore not one measurable quantity, and a model’s result on one task or category should not stand in for another. The tasks are still benchmark proxies and not agent traces.
When the needle stops looking like the question: NoLiMa Several familiar retrieval tests benefit from lexical overlap between a question and the evidence, but that is not a claim about every task in the preceding studies. Modarressi et al.’s NoLiMa (ICML 2025) minimises such overlap, so answering requires inferring a latent association. Thirteen models claiming contexts of at least 128K tokens were tested. They performed well below 1K tokens. At 32K, eleven fell below half of their own short-context baselines, and GPT-4o fell from 99.3 to 69.7 per cent. The authors report that reasoning-enhanced models and chain-of-thought prompting also struggled. They suggest that retrieving evidence without literal matches becomes harder as context grows, while noting that deeper mechanistic analysis lies beyond their scope. This book adopts the reported behavioural comparison, not a proven account of its internal cause.
NoLiMa also defines effective length as the largest tested length at which a model scores at least 85 per cent of its short-context base score. That is one operationalisation, and RULER’s “satisfactory” is another. This book uses effective length only with its benchmark and threshold attached, because it is a property of a model under a criterion and not a property the provider publishes.
Length, distractors and structure: the context-rot report Hong, Troynikov and Huber’s report “Context Rot” (Chroma, July 2025) tests eighteen models from four families on tasks whose difficulty is held fixed while input length varies. It is a company technical report and not peer-reviewed. It provides additional controlled evidence consistent with the peer-reviewed studies, and its model-specific results should be treated as technical-report findings until independently replicated.
Its manipulations are useful as categories. Distractors are topically related to the needle but do not answer the question, and irrelevant content is unrelated to both. In its needle extension, one distractor reduced performance relative to a needle-only baseline and four reduced it further, with effects varying across distractors and models, and unrelated filler also degraded performance. Related and unrelated added material therefore produced different degradation patterns, which is enough to keep the two manipulations apart. The report does not establish two internal mechanisms.
Three further results are bounded here. In a conversational-memory test, prompts containing only relevant material averaged about 300 tokens and full-history prompts about 113K, and models performed better on the focused versions, with reasoning modes narrowing but not closing the gap. The answer was present in both conditions, so this is presence without helpfulness, but the conditions differ in length and in the amount of irrelevant material together, so the comparison does not separate them. Models also performed better on shuffled haystacks than on coherent ones, which shows that structure in filler cannot be assumed neutral, though the authors leave the mechanism unresolved and the finding should not be read as “coherent filler harms”. Finally, the report describes different models failing in different ways, some abstaining and some answering confidently and wrongly. The durable lesson is for measurement: final accuracy can hide different failure signatures.
The authors state plainly that they do not explain the mechanisms behind the degradation.
Context rot is a phenomenon Context rot is used here as a descriptive name for a pattern: a chosen behavioural measure declines as input length increases under specified evaluation conditions. Anthropic’s documentation uses the term for accuracy and recall degrading as token counts grow. The pattern does not establish that length itself is the causal variable when distractors, position, structure or task difficulty change alongside it. Retrieval difficulty, distractors, position, representation, interference and model-specific behaviour remain candidate explanations. Anthropic’s engineering guidance frames context selection in terms of a finite attention budget; this book treats that as useful design language, not a measured quantity or a demonstrated mechanism.
The literature comparison, organised by what each study lets the reader conclude:
Evaluation Population Manipulation Outcome Supports Does not support Lost in the Middle (Liu et al., TACL 2024) Models of 2023; QA, key-value retrieval Position of relevant information Retrieval and QA accuracy Position can affect successful use of present information A universal U-shape, or effects on agents RULER (Hsieh et al., COLM 2024) 17 models; 13 tasks Length, task type and complexity Task accuracy Simple retrieval success did not carry over to harder tasks A single usable length per model HELMET (Yen et al., ICLR 2025) 59 models; 7 categories Length, task category Category scores and correlations Synthetic scores did not reliably predict downstream scores That all synthetic tests are uninformative NoLiMa (Modarressi et al., ICML 2025) 13 models claiming at least 128K Length with minimal lexical overlap Accuracy against short-context baseline Sizeable drops at 32K when literal matches are absent The authors’ attention explanation Context Rot (Hong et al., Chroma report, 2025) 18 models; 4 families Length, distractors, structure, focused vs full Task accuracy, abstention Related and unrelated additions degrade differently; structure is not neutral Mechanisms; independent replication Failure vocabulary Four terms are defined operationally. They are diagnostic dimensions and not mutually exclusive bins, and they are not causal mechanisms. A failure can involve several at once, since a stale item can also be a distractor, compaction loss can create absence, position can interact with interference, and interference can occur in a small context.
Overflow. The request exceeds a hard capacity constraint and is rejected, truncated or halted. It is an observable property, and it is Chapter 4’s territory. Bloat. Material whose removal or replacement preserves the required behaviour under the evaluation while reducing context resource use. That makes bloat counterfactual and measurable. Repetition, unused content and stable-but-unexamined items detected by the Chapter 2 instrument are bloat candidates, not proven bloat, since an item that looks unused can still alter behaviour. Interference. A degradation in behaviour attributable, under a controlled comparison, to adding or changing surrounding context while the task-relevant evidence and the evaluation target are held fixed as far as the design permits. Its manipulations include unrelated filler, topically related material and near-miss distractors. The term names the observed effect, not the internal mechanism that produces it. Position sensitivity. A change in measured behaviour when relevant content moves within the rendered context while its content is held constant and other changes are minimised and reported. A practitioner diagnostic taxonomy Breunig (2025) identifies four failure modes that practitioners encounter when context accumulates in agent sessions. They are presented here as a diagnostic framework and not as four independent causal mechanisms. The categories overlap: a poisoned summary may create a clash, trigger distracting retrieval, and cause repeated bad actions. They are useful descriptions, not a partition.
Context poisoning. Incorrect information enters context and influences later decisions. The DeepMind Gemini 2.5 technical report documents an agent whose goals section was corrupted with misinformation, causing it to pursue impossible objectives. The report characterises the repeated-action behaviour as anecdotal; it is not presented as a general threshold. Context distraction. Accumulated history interferes with planning and reasoning. The same report observes that beyond roughly 100,000 tokens the agent favoured repeating actions from its history rather than synthesising novel plans, and describes this as an anecdotal observation. A Databricks study of long-context RAG performance (August 2024) reports that Llama 3.1 405B’s average performance began declining beyond approximately 32K tokens in the studied configuration; this is a 2024 RAG finding, not a general agent-context limit. Context confusion. Irrelevant material, particularly unnecessary tool definitions, causes wrong decisions. The Berkeley Function-Calling Leaderboard documents differences in model performance across function-calling tasks of varying complexity, including multiple and parallel calls and irrelevance detection. These comparisons motivate investigation of tool-selection failures, but do not by themselves isolate the causal effect of exposing additional tool definitions for an otherwise identical task. Paramanayakam et al. (2025) demonstrate that a quantized Llama 3.1 8B fails a GeoEngine task with 46 tool definitions but succeeds with 19, within the same context window; this is a specific model and benchmark example, not a general threshold of 19 or 46 tools. Context clash. Conflicting information or previous assumptions disrupt subsequent reasoning. Laban et al. (2025) show that sharding a fully specified prompt into a multi-turn exchange, where the model makes premature attempts before having all information, degrades performance by the authors’ average 39% relative degradation across their six task families (15 models). The model anchors to its own early answers. These four descriptions map onto the book’s more precise vocabulary: poisoning is a provenance and status failure (Chapters 11, 15); distraction is interference and position sensitivity (this chapter, Chapter 6); confusion is capability-surface bloat and admission of irrelevant candidates (Chapters 14, 17); clash is authority and factual conflict (Chapter 19) and temporal inconsistency (Chapter 20). The taxonomy is a practical entry point; the book’s distinctions between selection, representation, assembly, authority and behavioural evaluation remain the analytical foundation.
Related dimensions belong to later chapters and add to this vocabulary without displacing it. Conflict between context items is Chapter 19’s, staleness is Chapter 20’s, and loss through removal or transformation is Chapter 11’s. This chapter studies added material.
Transfer to agents Much of the evidence reviewed here uses synthetic retrieval, question answering or other bounded benchmark tasks rather than coding agents operating with tools across a trajectory. HELMET includes application-oriented evaluations, but these are still benchmark proxies, not observations of the agent workflow this book intends to study. The following transfer claims are therefore hypotheses, not conclusions.
Book hypothesis. In coding-agent tasks, semantically similar near-miss material will degrade evidence use more than unrelated material of the same token count.
The basis is the Context Rot distinction between related and unrelated additions. What is speculative is that it transfers to agent-shaped bundles and that the size ordering holds. The hypothesis would be weakened if unrelated filler degrades performance as much as near-miss material, and not supported in the tested conditions if no added-material condition differs from the baseline. Staleness is a separate variable, taken up in Chapter 20, so near-miss values are used here in preference to superseded documents unless temporal validity is controlled.
Book hypothesis. Position sensitivity in agent-shaped bundles may differ from the static-document U-shape, because agent bundles are ordered chronologically and their parts have different semantic roles.
This is a prediction to test, not an established difference. The pattern could be a U-shape, primacy, recency, a monotonic trend, an interaction with content type, or no stable effect, and the design tests enough positions to find out which. The hypothesis is not supported in the tested conditions if no stable positional effect appears.
Proposed experiments Nothing in this section has been run. The companion repository holds a design, and execution is deferred. The two experiment families are separate: added-material type is the primary manipulation in A, and position is the primary manipulation in B. In both, the task fixture is the unit across which generalisation is assessed; repeated runs on one fixture estimate within-fixture variability and are not additional independent tasks. Report fixture counts, conditions, repeated runs, model versions and the analysis used to aggregate across fixtures. Randomise or counterbalance condition order where execution state could introduce order effects.
Experiment A: added-context interference. Hold fixed the task, critical evidence, model and version, settings, instructions and available tools. Compare a no-addition baseline with equal-length additions of unrelated filler, topically related non-answer material and near-miss distractors. Use a separately defined neutral-padding condition to estimate the effect of adding length without the targeted semantic relation; do not claim that neutral padding is behaviourally inert without testing it. Added tokens can shift the critical evidence’s normalised position even if its absolute position is fixed, so report both positions and, where feasible, include position-matched controls. Randomise conditions across task fixtures, and distinguish intended from observed token counts at the assembly boundary. Measure task success, unsupported claims, abstention and adoption of distractor content. Evaluate presence, locatability and usability with separately labelled probes rather than treating the probes as observations of the unchanged original task.
If a gradient appears, it supports transfer of added-context interference into the tested tasks. If all conditions perform alike, the synthetic-distractor evidence has not transferred under these conditions, and the book reports that and drops the claim. If unrelated filler harms as much as near-miss material, the distractor-specific motivation is weakened. If nothing degrades, that is the result.
Experiment B: position sensitivity. Hold the evidence text, surrounding material, model, settings and evaluation target constant while moving the critical evidence to several pre-specified normalised positions. Use fixed-length slots or position-preserving swaps where feasible, so movement does not also add or delete substantive material. Report actual rendered token positions, total lengths and unavoidable formatting changes; check whether moving the evidence changes its role, source wrapper or authority cues. Counterbalance conditions across fixtures and repeated runs, and test enough positions to distinguish plausible patterns rather than assuming a beginning-middle-end U-shape in advance.
If position matters, Chapter 6 gains empirical motivation from this experiment. If it does not, Chapter 6 must rest on other evidence and cannot cite it. If the shape differs by model or task, position policy must be conditional and not universal.
Inspect this in the Context Viewer The viewer scene for this chapter places two synthetic bundles side by side: a lean bundle of three items and a fuller bundle of six carrying the same critical evidence plus declared distractor and filler material. Open the scene and confirm what differs structurally. What to examine: which items the fixture declares task-critical, and how much of the fuller bundle they occupy. No outcome was measured, so present does not imply helpful here; whether more helps is Chapter 26’s question, not this scene’s answer.
The fuller bundle carries more; whether more helps is not measured here. Constructed illustration. Open the comparison scene in the Context Viewer
Where this leaves the problem Fit is necessary and nowhere near sufficient. The literature shows performance declining well below advertised limits under specific tests, and it shows that the size of the decline depends on task, model, condition and metric. The controlled studies support the claims that position can affect behaviour and that added surrounding material can reduce performance. They do not establish that the same holds for agent bundles, and the book’s own experiment has to test that before any transfer becomes a book result.
The chapter’s closing chain extends Chapter 4’s “can fit ≠ should admit ≠ will help”:
present ⇏ locatable ⇏ usable ⇏ helpful
If position can matter, it is a candidate control variable, and order is something a system either decides or lets happen. That is the next chapter’s subject:
Even if we know which information belongs in the bundle, where should we put it?
References Kamradt, G. “Needle In A Haystack.” Third-party evaluation repository, original runs November 2023. Origin of the needle test. https://github.com/gkamradt/needle-in-a-haystack Liu, N. F., Lin, K., Hewitt, J., et al. “Lost in the Middle: How Language Models Use Long Contexts.” Transactions of the Association for Computational Linguistics 12 (2024): 157–173; arXiv:2307.03172 (2023 preprint). Peer-reviewed. https://arxiv.org/abs/2307.03172 Hsieh, C.-P., Sun, S., Kriman, S., et al. “RULER: What’s the Real Context Size of Your Long-Context Language Models?” COLM 2024; arXiv:2404.06654. Peer-reviewed. https://arxiv.org/abs/2404.06654 Yen, H., Gao, T., Hou, M., et al. “HELMET: How to Evaluate Long-Context Language Models Effectively and Thoroughly.” ICLR 2025; arXiv:2410.02694. Peer-reviewed. https://arxiv.org/abs/2410.02694 Modarressi, A., Deilamsalehy, H., Dernoncourt, F., et al. “NoLiMa: Long-Context Evaluation Beyond Literal Matching.” ICML 2025; arXiv:2502.05167. Peer-reviewed. https://arxiv.org/abs/2502.05167 Hong, K., Troynikov, A., Huber, J. “Context Rot: How Increasing Input Tokens Impacts LLM Performance.” Chroma technical report, 14 July 2025. Not peer-reviewed; open codebase. https://www.trychroma.com/research/context-rot Anthropic. “Context windows.” Official documentation, verified September 2026. Context rot named; accuracy and recall degrade as token count grows. https://platform.claude.com/docs/en/build-with-claude/context-windows Anthropic Applied AI team. “Effective context engineering for AI agents.” Published 29 September 2025. First-party engineering guidance; the finite attention budget framing is its own. https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents Breunig, D. “How Long Contexts Fail.” 22 June 2025. Practitioner synthesis; four failure modes and mitigation taxonomy. https://www.dbreunig.com/2025/06/22/how-contexts-fail-and-how-to-fix-them.html