Part 3 — Give Intelligence a Runtime

The question the acceptance could not answer

Chapter 14 ended with a task that was provably complete. The acceptance cited the task’s criteria, the call, its final attempt, the interpretation, the exact artifact bytes and the checks run on them.

It did not cite what the model was shown.

Ask most systems that question and you get one of two answers. The first is “the conversation so far”: everything that happened, in the order it happened. The second is “whatever was in the prompt”: a string someone assembled and nobody recorded. Neither answer can be checked afterwards, and neither was decided on purpose.

What exactly was the model allowed to see?

This chapter answers a narrower question than it sounds like it should, and the narrowing is the result.

A transcript is an order you did not choose

A transcript is a context policy by default. It includes everything, in arrival order, until something runs out of room.

Two results suggest why that is a poor policy, even before cost enters.

Liu and colleagues placed the document containing an answer at different positions in long inputs and measured how well models used it (Liu et al., 2024). Performance was often highest when the relevant information came at the beginning or the end, and degraded when it sat in the middle, including for models built for long contexts. Where material lands is part of the input.

Shi and colleagues added irrelevant sentences to grade-school arithmetic problems and built a benchmark from them, GSM-IC (Shi et al., 2023). Accuracy fell markedly when the irrelevant information was present. Sampling several answers, and telling the model to ignore irrelevant information, were among the mitigations that helped. What you include is part of the input too, including what you should have left out.

Both findings are bounded by the models, tasks and prompts they tested. This chapter runs no model and measures neither effect. They are the motivation, not the evidence: if position and irrelevant material change what a model does, then which material went in, and in what order, is a decision worth making deliberately and recording.

So stop treating context as history. Treat it as an input to be compiled.

Five states of a piece of material

Take any piece of material a model might use: an objective, a source paragraph, an earlier observation, a claim derived from another call. Before a model can use it, five different questions have to be answered, and they have different answers:

available   it exists: in the ledger, the artifact store, the claim record
    ≠
eligible    the operation is permitted to use it (no seal forbids it)
    ≠
selected    requirements and the budget admit it into the package
    ≠
rendered    its bytes were actually placed in the request
    ≠
received    the provider processed those bytes

“Memory” is what is available. “Context” is what is selected for one operation. Everything in between is a decision, and exclusion is evidence: if something available was left out, the record should say so and say why.

The last state is not observable from this side of the API. The fourth is observable, and as this chapter will show, it is exactly where CodeAI’s current record stops.

The compiler

CodeAI’s ContextCompiler already existed. It was not rebuilt for this chapter. What it does, read from the source at commit fe0797d:

• Candidates, not history. Each piece of material is a ContextCandidate with a kind (event, artifact, claim), an ID, a declared size, a required flag and the lineage IDs it derives from.
• A fixed order. Candidates are sorted by kind and then ID. The order they arrive in does not matter.
• Required first. A required candidate that was not supplied, or that a seal forbids, raises RequiredContextMissing. Required candidates whose declared sizes exceed the budget raise ContextBudgetUnsatisfiable. Neither failure produces a package.
• Seals by lineage. A Seal lists forbidden event, artifact, call and lineage IDs. An optional candidate whose own ID or declared lineage matches is excluded, with the reason.
• Budget for the rest. Optional candidates are admitted in the fixed order while they fit; the rest are excluded “because budget”.
• Two identities. The package ID is a SHA-256 over the task, actor, prompt and version, objective, budget, seal and the sorted selected IDs. The trace ID is a SHA-256 over the package ID and every decision with its reason.

Runtime.compile_and_record_context compiles and appends a context.compiled event carrying the package and its trace.

The pipeline, as a flow of decisions:

    flowchart TD
    CAND["candidates<br/><i>event · artifact · claim, with size + lineage</i>"] --> REQ{"required?"}
    REQ -->|"yes"| RQ["required check<br/><i>missing or sealed → raise,<br/>no package</i>"]
    REQ -->|"optional"| SEAL{"seal forbids<br/>id or lineage?"}
    SEAL -->|"yes"| EX["excluded<br/><i>with the reason recorded</i>"]
    SEAL -->|"no"| BUD{"fits the budget?"}
    BUD -->|"no"| EX
    BUD -->|"yes"| SEL["selected<br/><i>fixed order: kind, then id</i>"]
    RQ --> SEL
    SEL --> PKG["context package<br/><i>id = hash of inputs + selection</i>"]
    PKG --> TR["compilation trace<br/><i>every decision with its reason</i>"]
  

That is a claim about behavior. The experiment checks it.

The experiment

The demonstration was preregistered before it ran. Its hypotheses, fixture, budget and controls were written to preregistration.json at 18:52 UTC, and the execution record pins that file’s hash, the CodeAI commit and the hash of every CodeAI source file used. No model was called, and outbound connections were blocked.

The fixture has five candidates with declared sizes. These are synthetic size estimates, not tokenizer counts:

The seal forbids call lineage sibling-call. This is the isolation Part 5 depends on: an independent call must not see what its sibling produced. The budget is 16.

H1: the order candidates arrive in does not matter

The same five candidates were supplied in four different orders: A B C D E, D A C B E, C B D A E, and B D A C E.

Every run selected A, B and D, for a total of 13. Every run excluded C “because budget” (13 + 5 would be 18) and E because the seal forbids call lineage sibling-call. All four produced package ID cfa87a17… and trace ID 5cb3a3ec….

Four permutations of one fixture are not a proof for all inputs. They do show that, for this input, nothing about arrival order leaked into what was selected or how it was identified.

H2: the seal excludes sibling-derived material, by lineage

At a budget of 100, so that budget excluded nothing, the sealed fixture selected A, B, C and D (total 18) and excluded E with a reason naming sibling-call.

The paired probe removed only E’s declared lineage. With nothing linking it to the sibling call, E was admitted, and the total became 25.

Both halves matter. The seal worked, and it worked on what it was told. The compiler does not look at content and infer where it came from. If lineage is not declared, the seal cannot see it.

H3: required material that does not fit is a failure, not a trim

Required A and B were given sizes totalling 10 against a budget of 9, and the compiler raised ContextBudgetUnsatisfiable (“required 10 tokens exceed budget 9”) without producing a package or trace. It did not silently drop the source to make room, which would have left a model to produce an answer without ever seeing its source.

H4: required material that is not there is a failure

The request required an ID that was never offered, and the compiler raised RequiredContextMissing, naming the missing ID, without producing a package. A supplementary case required E itself while E was sealed; that also failed, with the message naming the forbidden lineage.

Recorded, then reopened

The runtime path, compile_and_record_context, compiled the fixture at a budget of 8 and appended one context.compiled event. A separate operating-system process then opened the same SQLite ledger and artifact store, and the recovered package, trace and events matched the original while the inspection added no events.

The two runtime failure cases (a required item over budget, and a missing required item) left the ledger at the same event count before and after: 5 and 5. A failed compilation appends nothing to the ledger. The request and the exception exist only because the experiment’s own harness kept them.

What the identity binds, and what it does not

A package ID looks like a fingerprint of the context. The experiment tested that impression directly, changing one thing at a time against the same fixture:

The last row is the important one. The objective’s text changed from “Review the paragraph using its source.” to “Changed under same ID”, and both identities stayed exactly the same.

The package ID is a selection identity. It names which IDs were chosen under which task, actor, prompt, seal and budget. It is not a content hash of the material behind those IDs.

A second probe pushed on the same seam. A required artifact ID with no bytes behind it (ghost-artifact) was offered as a candidate. It was selected, “included because required”, and the package was produced. Looking the ID up in the artifact store failed. Offering an ID is not the same as proving its bytes exist, and the compiler checks only the first.

Allowed to see is not what was sent

Now the boundary this chapter is really about.

The experiment prepared, without sending, the request that would carry the recorded package. The prepared body was:

{"model": "fixture", "messages": [{"role": "user", "content": "Review selected inputs."}]}

The selected source paragraph B, whose text begins SOURCE-B:, does not appear in it. Neither does the objective or the derived claim. The same harness then read sealed artifact E straight from the artifact store, which worked: the seal had governed the package, not the store.

Reading the adapter source confirms this is not an accident of the fixture. At fe0797d, the OpenCode adapter builds the request text from two fields: the call’s instruction and the package’s prompt string. Nothing dereferences the selected event, artifact or claim IDs and renders their contents.

So, in CodeAI as it stands:

• Available, eligible and selected are decided deterministically, recorded with reasons, identified by hash, and recoverable by another process.
• Rendered is whatever the prompt string contains. The package’s selection is not what put material into the request.
• Received is not observable at all.

A context package is a record of permission. It becomes a record of input only when the bytes sent can be traced back to it, and at fe0797d they could not.

Current practice for agents sharpens this distinction rather than dissolving it. Anthropic’s guidance on context engineering treats context as a finite resource with an attention budget, aims for the smallest set of high-signal tokens, and names techniques for long tasks: just-in-time retrieval, compaction (summarizing a conversation near the window limit and starting again from the summary), structured note-taking kept outside the window, and sub-agents that hand back condensed summaries (Rajasekaran et al., 2025). Those techniques decide what is in a window. A package records something else: what was available, what was excluded and why, under which identity. Compaction discards; a record preserves. A well-compacted transcript is still not an account of what the model was allowed to see.

Lampson described the general shape of this problem in 1973 (Lampson, 1973). His customer runs an untrusted service and grants it access only to the items it needs; the question is whether information can still move by some other route. His examples run from files the service may write, through the bill for the service, to the load it puts on the system. His conclusion for a trustworthy supervisor was blunt: It is necessary to enumerate them all and then to block each one.

The mapping onto context is this chapter’s, not his: the seal blocks one channel, the package, and only for lineage someone declared. The artifact store is another channel, with no block on it, while the prompt string is a third that bypasses selection entirely. Isolation claimed on the strength of the package alone is isolation of one enumerated channel.

Closing the gap

The boundary this experiment found was closed in a follow-up stage, Stage 15B, at CodeAI 0f9a83b, and the closure is worth reading as closely as the gap. It is opt-in: a call that sets context_render = "context-render-v1" has its selected material rendered by the runtime, and every other call behaves exactly as before.

For an opted-in call, the runtime resolves each selected ID to bytes: artifacts from the content-addressed store with their integrity checked, events as their payload in the ledger’s canonical serialization, claims as their recorded payload. An ID that cannot be resolved raises before anything is prepared, and the call records call.preparation_failed with no manifest and no attempt. Resolved items are laid out in the same canonical order the compiler uses, each with a header carrying its content hash and length, and the instruction, context and query are composed into one input. Before any provider effect, the runtime checks that the prepared body carries exactly that text, stores the rendered bytes as an artifact, and writes the binding into the manifest. From the preserved Stage 15B manifest, trimmed:

{
  "context_package_id": "7cfdf28e…",
  "context_render_version": "context-render-v1",
  "rendered_context_sha256": "a97141e8…",
  "rendered_items": [
    {"kind": "artifact", "item_id": "eae62863…", "content_sha256": "eae62863…", "start": 178, "end": 264},
    {"kind": "claim",    "item_id": "D",         "content_sha256": "2e86fbe0…", "start": 394, "end": 681},
    {"kind": "event",    "item_id": "A",         "content_sha256": "7c73dc25…", "start": 810, "end": 859},
    {"kind": "event",    "item_id": "C",         "content_sha256": "edb5d538…", "start": 988, "end": 1029}
  ],
  "input_layout": [
    {"part": "instruction", "start": 0,    "end": 51},
    {"part": "context",     "start": 53,   "end": 1099, "sha256": "a97141e8…"},
    {"part": "query",       "start": 1101, "end": 1143}
  ],
  "request_body_sha256": "25183509…"
}

The five states now have five separate answers. The package ID is still a selection identity. Each rendered item carries a content hash, so source bytes are named. The rendered context has its own hash and stored bytes. The request body hash covers the prepared request whose text carries those bytes at the recorded offsets. And receipt by the provider is still not observable from this side.

The preregistered stage re-ran this chapter’s awkward cases against the new path. The source paragraph B’s 86 bytes appeared in the request at their recorded offsets, and sealed E was absent from both the rendered bytes and the request text. Rewriting event A’s payload under the same ID left the package ID unchanged, as before — but now the changed item was named as exactly A, and both the rendered hash and the request hash changed. The ghost artifact failed resolution with zero transport calls and a recorded preparation failure. Permuted supply order produced identical package, rendered and request hashes, and a control call without the opt-in sent exactly the instruction and query. An independent verifier passed all 59 of its semantic claims and rejected five seeded corruptions, including B’s bytes removed and sealed E inserted with hashes and offsets recomputed to match.

The limits are part of the result. Only calls that opt in are covered, and paths that don’t — including fan-out, as Chapter 23 will show — still send the prompt string. One adapter and one API protocol were exercised. The branch that refuses a prepared body not carrying the composed text is implemented but was not exercised. Events and claims render as raw payload JSON, a provenance format rather than a prompt design. Seals are still not access control. And the transport was synthetic: the harness recorded bytes and hashes without measuring what a model does with the rendered context.

Checking it without trusting it

The bundle’s verifier imports neither CodeAI nor the program that produced the evidence. It reads the raw requests and results and the reopened events. From those it:

• recomputes both identities from their fields, checks the arithmetic, the ordering and every exclusion reason
• confirms the required failures left the ledger unchanged
• checks persistence, and that the preregistration predates execution
• checks every preserved file against the hash inventory

All 45 of its claims pass.

A verifier that always passes proves nothing, so four corrupted copies of the evidence were prepared, each run with the byte inventory deliberately bypassed so that only the semantic checks could catch them:

Each exited with a failure, naming exactly those claims. For this chapter the verifier and all four corruptions were rerun, and both identities were recomputed for the four permutations from their raw fields; the results matched.

Where it is still weak

1. Selection reaches the request only by opt-in. At fe0797d the adapter rendered only the instruction and prompt string. Stage 15B binds rendered selected bytes to the request for calls that opt in; every other path still sends the prompt string.
2. The package identity is not a content hash. A payload rewritten under the same ID, a changed declared size, or added metadata leaves the package ID unchanged. For opt-in calls, per-item and rendered hashes now name the bytes beside it.
3. Sizes are declarations. The fixture’s sizes are synthetic. On the convenience path, artifacts and claims count as zero, and event sizes are estimated from serialized length (about four characters per token), not measured by a tokenizer.
4. Seals see only declared lineage. Remove the lineage and sibling-derived material is admitted. The compiler never infers provenance from content.
5. Seals are not access control. Sealed material remained readable directly from the artifact store.
6. IDs are not checked for existence. A required artifact ID with no bytes behind it was selected.
7. Failures leave no runtime record. Missing and over-budget compilations raise before anything is appended; the evidence of them exists only in the experiment’s harness.
8. Reopen covers success only. The recorded event carries the selection and its trace, not the full offered inventory with every requirement, size and lineage.
9. One fixture, no model. Four permutations of five candidates, and nothing here measures whether selection changes an answer.

Do this now

Forty minutes. Find out what your model actually saw.

1. Take the last model call your system made that mattered. Write down three lists: what material was available, what was selected for the call, and what text was actually in the request. If you cannot produce the third list from a record, you do not know what the model saw.
2. Find where your system builds the prompt. Does it render from a recorded selection, or from whatever variables were in scope?
3. Supply the same inputs in a different order and check whether the request or any recorded identity changes.
4. Change the content of one source without changing its ID or filename and see what notices the difference.
5. Mark one input as required and remove it. Does the call fail, or run without it?

If you are building with an assistant:

Treat context as a compiled input, not accumulated history.
- Represent each piece of material as a candidate with kind, id, declared
  size (label it estimated unless measured), required flag and lineage ids.
- Order candidates canonically. Include required candidates first; raise if a
  required candidate is missing, sealed, or exceeds the budget. Never trim
  required material to fit.
- Exclude optional candidates by seal (declared lineage) or budget, and record
  a reason for every exclusion.
- Hash a package identity over the selection and its parameters, and a trace
  identity over every decision and reason. Say plainly that these are selection
  identities, not content hashes, unless you also hash the material's bytes.
- Record successful compilation durably and reopen it from another process.
- Then close the gap this chapter found: render the request only from the
  recorded selection, and record a hash of the rendered request next to the
  package so "selected" and "sent" can be compared.
- Test: permute inputs (identities stable), remove declared lineage (seal
  admits), required over budget and required missing (no package), payload
  changed under the same id (identity unchanged unless bytes are hashed).

Failure modes

• Context as transcript. An order nobody chose, and content nobody selected.
• Silent trimming. Required material dropped to fit a budget; the answer arrives anyway.
• Order-dependent selection. The same inputs in a different order produce a different context.
• Unrecorded exclusion. Something available was left out and nothing says why.
• Treating a selection hash as a content hash. The source changed and the identity did not.
• Trusting undeclared provenance. The seal only blocks lineage someone wrote down.
• Calling a seal access control. The sealed bytes were still in the store.
• Recording the selection, sending something else. The package said one thing; the request carried the prompt string.

What this chapter established

• What a model was allowed to see has five distinct states: available ≠ eligible ≠ selected ≠ rendered ≠ received. Exclusion is evidence.
• Position and irrelevant material have been shown to change model behavior in other work (Liu et al.; Shi et al.). That motivates deliberate, recorded selection; this chapter measured no model.
• CodeAI’s existing compiler, preregistered and executed at fe0797d, behaved as specified for the fixture:
  • Four input orders gave one selection and identical package and trace identities.
  • The seal excluded sibling-derived material by declared lineage, and admitted it when the lineage was removed.
  • A required item over budget, and a missing required item, each failed with no package.
  • Successful compilation was recorded and recovered by another process without adding events.
• The package ID is a selection identity. Rewriting a payload under the same ID, changing a declared size or adding metadata left it unchanged, and a required ID with no bytes behind it was selected.
• At fe0797d, selected material did not reach the request: the prepared body contained only the prompt string, and sealed material stayed readable from storage. The package recorded permission, not input.
• Stage 15B (0f9a83b) closed that boundary for opt-in calls: the runtime renders selected bytes, binds per-item hashes, rendered hash and layout into the manifest before any effect, and records resolution failures; an independent verifier passed 59 claims. Other paths still send the prompt string.
• An independent verifier passed all 45 claims and rejected four seeded semantic corruptions, naming the claims each one broke.

Next

The package can be recovered after a clean exit, as long as the compilation succeeded. The rest of what the decision depended on cannot. The full list of what was offered, what was required, the declared sizes, the lineage, and every attempt that failed, lives only in this experiment’s harness.

A process that picks up this work tomorrow would know what was selected, not what it was selected from, or what was tried and refused. Continuing an investigation needs more than the last answer. It needs the working state.

Continue with Externalize Working Memory.

References

• Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. Lost in the Middle: How Language Models Use Long Contexts. Transactions of the Association for Computational Linguistics, vol. 12 (2024), pp. 157–173. https://doi.org/10.1162/tacl_a_00638
• Freda Shi, Xinyun Chen, Kanishka Misra, Nathan Scales, David Dohan, Ed H. Chi, Nathanael Schärli, and Denny Zhou. Large Language Models Can Be Easily Distracted by Irrelevant Context. Proceedings of the 40th International Conference on Machine Learning, PMLR 202 (2023), pp. 31210–31227. https://proceedings.mlr.press/v202/shi23a.html
• Butler W. Lampson. A Note on the Confinement Problem. Communications of the ACM, vol. 16, no. 10 (October 1973), pp. 613–615. https://doi.org/10.1145/362375.362389
• Prithvi Rajasekaran, Ethan Dixon, Carly Ryan, and Jeremy Hadfield (Anthropic Applied AI team). Effective Context Engineering for AI Agents. Anthropic Engineering, 29 September 2025. https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents

Implementation sources: CodeAI fe0797d for the Stage 15 experiment (unchanged by it), and 0f9a83b for Stage 15B (src/codeai/rendering.py: render_context, compose_model_input; CallSpec.context_render; manifest binding and call.preparation_failed in src/codeai/runtime.py). src/codeai/context.py: ContextCompiler.compile_with_trace, ContextCompiler.compile_candidates, ContextCandidate, CompilationTrace, candidate_blocked_by_seal, event_lineage, _estimate_tokens. src/codeai/domain.py: ContextPackage, Seal. src/codeai/runtime.py: Runtime.compile_and_record_context. src/codeai/providers.py: _prompt_messages, which builds request text from the call instruction and the package prompt. Focused tests (tests/test_context.py, tests/test_epistemic.py): 29 passed; full suite: 289 passed. Stage 15B evidence: experiments/applied-ai/evidence/context-rendering/2026-09-13-0f9a83b/ (preregistration, seven cases, preserved manifest and rendered bytes, independent verify.py with 59 semantic claims and five seeded corruptions, chapter-evidence-report.md). Stage 15 evidence: experiments/applied-ai/evidence/context-selection/2026-09-13-54e32384/, containing the preregistration and execution record, fixture, case requests and results, the runtime ledger with its reopened export, identity counterexamples, the downstream boundary probe, the independent verify.py, four seeded corruptions, test outputs, chapter-evidence-report.md and hashes.json. Producer: experiments/applied-ai/context_selection_demo.py (the executed copy is pinned as the bundle’s run.py).