When Memory Becomes Documentation

The book’s definition forces a question it has not yet asked.

Memory is when retained past experience changes what the system does now.

Every mechanism so far has assumed that retained experience stays in the form it was stored. Retrieved history is retrieved. Derived state is rebuilt. Trust is re-evaluated on the way into context. But at some point a system stops treating an observation as evidence about the past and starts treating it as an instruction about the present. A remark becomes a rule. A workaround becomes a policy. A decision becomes a specification that future work must satisfy.

Nothing in the book’s architecture covers that transition, because nothing in it was designed to.

In October 2026 an argument arrived that covers it badly and by force. Kevin Liao’s essay Agents Don’t Need Memory. They Need Documentation states that memory plugins are solving the wrong problem, that the entire plugin ecosystem is one similarity search wearing different costumes, and that the correct answer is ordinary Markdown with a consult-build-update loop. The essay is crisp, well argued, and directly aimed at this book’s subject matter.

It is also, on its own evidence, an argument rather than a measurement. That does not make it wrong. It makes it a challenge, and this chapter takes it as one.

The claim, stated at full strength

The essay’s case against recall-shaped memory has five parts, and they deserve to be restated without weakening before being examined.

Similarity establishes nothing about currency or correctness. A ranking returns whatever sits nearest in embedding space, which is a statement about vectors rather than about the project. Extraction discards what a snippet cannot hold. Motivation, constraints, and the environment in which an observation held are all lost before any search runs. The past is treated as current, because code changes daily and a captured snippet does not. Agents cannot search for a gap they do not know they have, since noticing a missing fact and phrasing a query that fills it are both requirements a retrieval tool imposes on a system that does not yet know it is missing anything. And the store is unauditable, because ten thousand embedded rows cannot be enumerated, staleness-checked, or reviewed by a person.

The essay’s proposed replacement is a workspace of Markdown documents — instructions, specifications, decisions, research, indexes — consulted before work and updated after it.

The argument has force because each of the five complaints is really a restatement of something this book measured. Chapter 3 established strong retrieval as a serious baseline rather than a strawman, and that result survives untouched by this essay. Chapter 8 established that temporal validity must be handled explicitly. Chapter 7 established that derivation lineage is not evidential support. Chapter 17 established that retention grants no authority.

What the essay adds is a claim about category: that these are not tuning failures within a sound architecture but symptoms of choosing the wrong one.

What the essay does not establish

The headline does not survive contact with the evidence, and the book’s own results are the reason.

Nothing in the essay measures anything. There is no baseline comparison, no ablation, no corpus, no failure taxonomy. It is an author argument supported by an author’s usage, and the implementation’s own comparison document generalises it into a claim about an entire market — that every memory plugin works the same way — without a source. That kind of claim is exactly the kind this manuscript has spent its whole length refusing to accept.

More decisively, the book’s Chapter 12 run contradicts the strong form directly. Strong hybrid retrieval scored 0.393 against a no-memory floor of 0.226, and removing decisive memory dropped the six applicable tasks from 0.778 to 0.250. Retrieval is not a broken category. On controlled diagnostic tasks it does most of the work.

So the honest position is neither agreement nor dismissal. Documentation is one durable representation among several, and the claim that it is the representation is untested. What makes the essay worth a chapter is not its conclusion but the pressure it applies to a specific sentence in the manuscript.

The sentence this chapter is about

The book’s persistence claim, stated in the final synthesis, is short and load-bearing:

Derived state may be worth persisting for cost, but persistence does not automatically earn epistemic authority.

Documentation is the most persuasive persisted representation there is. It is human-readable, diffable, version-controlled, reviewable. When an agent writes a document and a person approves it, it looks exactly like knowledge.

That is precisely why the sentence needs testing here rather than in an appendix.

What the implementation actually does

The essay ships with a working system, which makes it a case study rather than an opinion. Operator Memory is an open-source documentary workspace: BSD-3-Clause, roughly 180 stars, about ninety commits, adapters for several coding agents including OpenCode and Pi. Everything below is read from the project’s own README and architecture documents on 4 October 2026. It is vendor-reported and was not line-audited, which limits what can be claimed from it.

Three features matter more than the rest.

Partitions with an explicit conflict hierarchy. Knowledge lives in a private project partition, a shared partition published with the repository, and a personal cross-project partition. The documented precedence when instructions conflict is private over user over shared.

This is the most interesting thing in the design, and it is not about retrieval at all. It answers the question which durable artifact wins, not which text is similar. A system that has decided there is a correct answer to conflicting durable instructions has accepted that authority is structured, and that structure needs a resolution rule. This is the closest the external design comes to the problem this chapter is about.

Deterministic orientation. Each session loads a fixed preamble: framework guidance, standing instructions merged by authority, a compact catalog per partition, a project index, and repair diagnostics. The claim is that identical session starts produce identical orientation. Nothing is ranked or guessed at startup.

That is a real answer to the essay’s first complaint, and it is worth being precise about what it answers. It removes unreliable ranking from session start. It does not remove retrieval from the session.

The catalog admission. And here the design concedes more than its marketing does. Freeform documents are reached through catalog guidance chosen by the agent, explicitly not by automatic selection. Catalogs map knowledge; indexes map code.

So the honest reading of this system is narrower than the essay’s framing. Documentary memory replaces an unauditable ranking failure with an agent-mediated reading of an index. The failure it removes is that a memory store cannot be examined. The failure it retains is an agent not consulting the map — which is Chapter 10’s problem, wearing different clothes.

The architecture document also concedes staleness outright, stating that the workspace can go stale after significant upstream change, and that the design’s answer is to keep that failure in ordinary files where a person can find and correct it, with a repair skill shipped alongside. That is an honest position and a notable one: the failure is not prevented, it is made findable.

Finally, a caution about reading vendor material. The roadmap lists an “Observation Engine” for learning durable user observations, deliberately separated from explicit user instructions. It is not shipped. Treating a roadmap entry as a capability is how capability claims inflate.

What the community added, and what it did not

The Hacker News thread on the essay drew 230 points and 130 comments. It is read here in full, and it is community discussion rather than evidence — there is not one controlled comparison or measured result anywhere in it. Most operational claims are single-user anecdote, several comments promote the commenter’s own tool, and per-comment vote scores were not available in the retrieved form.

Used carefully, it is useful for two things: what practitioners report breaking, and where they disagree.

Two themes recur, and each is reported here as a single practitioner’s account rather than as an observed rate. The first is that documentation does not imply obedience: spike021 describes a standing preference for jq over an ad-hoc Python parser that the agent did not follow, locknitpicker generalises this to agent-authored skills treated as suggestions to be routed around, and yulaow argues that placement barely matters, because instruction text in a conversation, a skill, a hook, or an instruction file all decay after a few iterations. The remedy proposed across that cluster is deterministic enforcement rather than better prose.

The second is agent-authored authority. mzhaase describes an agent that grew decisions it was never asked for, escalated into prohibition phrasing, and cited in review a rule that had never been approved. pornel reports agents extrapolating unrequested detail into decision records. Two reports, from two users, of the same mechanism — which is a reason to design for the failure, not a measurement of how often it occurs.

And the thread is genuinely divided, which a chapter must not flatten:

• zahrevsky disputes the essay’s central analogy directly, arguing that snippets are the meeting notes and the real fault is note-per-sentence volume.
• locknitpicker tells bushido that versioned principles reinvent instruction files; bushido refuses, arguing that principles authorise a decision while code merely records one.
• Two practitioners report opposite scaling experience with the same idea: mzhaase a decision count in the hundreds, bushido struggling past two dozen.
• DriverDaily argues documents cannot be queried relationally and a database is required; apsurd rejects the comparison.
• jon-wood argues hooks must be deterministic; fxtentacle argues that reinforcement learning from verifiable rewards already does this and self-built hooks are a workaround.

stbenjam is the thread’s most sceptical substantive voice, arguing that agent-written documents grow large, that nobody can show they help, and that they rot — and attributing the silence to evaluation being hard. jmtulloss puts the same point as a rule: evals, or it did not happen.

The thread also contains the counter-pressure this chapter most needs. jdw64 invokes Peter Naur to argue that documentation cannot preserve a full model of a system, and suggests a system without the human capacity to hold theory implicitly may need a different route. One comment, no supporting argument, and no corroboration anywhere else in the thread. It is still the sharpest objection available.

Promotion

Behind the essay’s complaint sits an operation this book has not named.

Book hypothesis. Promotion is the controlled transformation of retained experience into a more durable representation intended to influence a broader class of future situations.

One warning about the word, because the book already uses it. Chapter 16’s “backtest before promotion” and the synthesis’s replay-before-promotion fallback both mean deploying a measured policy change into the architecture — promotion in the sense of shipping. What this chapter names is different: turning one retained observation into a durable representation that will govern future situations. Chapter 18 uses the second sense throughout. Elsewhere the older sense stands, and where the two could be confused, the sentence says which is meant.

The two are related and should not be confused into one idea. A policy change is replayed against earlier cases before it is adopted into the architecture. An observation is checked for scope and licence before it is promoted into durable knowledge. The first is the book testing itself; the second is the book deciding what its own system is allowed to believe.

The definition is provisional and the experiment designed to test it has not been run. What can be said now is that the transition has a property no other step in this architecture has: it changes authority, not just representation.

A note promoted to a principle is not a better-formatted note. It now constrains a class of future situations it was never present for, and it does so whether or not the situation resembles the episode that produced it. That is the whole hazard, and it is invisible in a system that only ever retrieves.

Promotion is also not a ladder that everything climbs. Some things should stop almost immediately.

experience
   ├── irrelevant or expired        → suppress, or keep as history only
   ├── useful only to this task     → task note
   ├── useful as evidence           → episodic memory
   ├── personal, never shared       → private user scope
   ├── stable shared knowledge      → documentation
   ├── an accepted choice           → decision record
   ├── a standing rule              → policy
   └── an invariant                 → executable enforcement

And promotion does not increase truth. A policy can be confidently wrong. A specification can faithfully encode a misunderstanding. An executable constraint can perfectly enforce the wrong rule. This is Chapter 16’s claim that something more useful is not thereby more true, made structural — which is why promotion needs a recorded licence rather than a recorded conclusion.

Chapter 1 already sketched this territory, running from historical event to interpreted memory to current belief to generalised principle to present action. This chapter is the rung between belief and principle, and it inherits that framing rather than inventing one.

The promotion problem

Consider an ordinary remark:

Use jq here; it’s already installed.

What did the speaker mean?

this command only
this task only
this repository
this class of JSON work
all future JSON parsing
all projects, everywhere
a general preference for tools already present

A retrieval system cannot fail here, because it never has to decide. It stores the sentence and may or may not surface it later. A promotion system must decide, and the ambiguity above is not resolvable from the sentence alone.

This is the structural difference between memory and documentation, and it is worth stating plainly: the scope of an instruction is not a retrieval property. It is a licence someone grants.

Chapter 17 already built the mechanism for exactly this, and the book should reuse it rather than invent a parallel one. The trust gate makes admission categorical, with a stated reason code, and gives trust precedence over usefulness. Promotion needs the same shape — a stated reason, a recorded approver, and a route by which the promotion is refused.

Failure classes

The book’s taxonomy keeps pairs apart, because conflating them is how a system gets credited for fixing something it did not. Promotion extends that pattern. Each entry is a new pair, not a replacement:

• Durable knowledge never promoted, versus useful knowledge promoted too far.
• The right rule, promoted to the wrong audience, versus the wrong rule promoted correctly.
• A promotion with no licence, versus a promotion whose licence has expired.
• A stale promotion still in force, versus a supersession that was applied but never recorded.
• A correct historical record deleted because it went out of date, versus a current record that should have superseded it.
• A durable claim with traceable evidence, versus a durable claim with no evidence at all.

That last pair is Chapter 7 arriving again. A document generated from three passages is one observation and two echoes, and generation makes this look like authorship. Promotion must not sever the path back to experience.

A worked promotion

The book’s running example is ready for this. World A moves an event store from SQLite to PostgreSQL across 2024, and the canonical artifacts already exist.

The progression, in the book’s own canonical artifacts:

benchmark (evt-203) → incident (incident-021) → proposal (session-031)
      → decision (adr-007) → staged implementation → production (fact-302)
      → derived restatement (runbook-006) → completed consequences

The contention evidence accumulates. A benchmark confirms it, then an incident confirms it for real, and a decision record eventually states that new event-store work should target PostgreSQL. That decision is still current in the canonical history; nothing supersedes it.

Each step is individually defensible. The failure is in the aggregate: a derived restatement reads exactly like an authoritative one, and nothing in the document says which it is.

That is Chapter 7 again, in a new costume. runbook-006 echoes adr-007; it is not independent support, and a reader who cannot tell the difference will treat repetition as corroboration.

At the point where the incident becomes a decision, something is gained: the reasoning is now available without reconstructing it, and the scope is stated rather than inferred. Something is lost: a later deployment-documentation update records no explicit link to the migration consequence it completes, so a reader of that document cannot see what remains outstanding. And something changes that neither gain nor loss describes — the decision now governs, rather than merely recording.

The instinct at that point is to extract a general principle. The candidate is:

Test concurrent-write behaviour before choosing embedded storage
for shared write-heavy services.

This is the step that needs an answer, and the book’s earlier chapters already supply two.

Chapter 1 shows the book reaching for this generalisation itself. Chapter 16 supplies the check: a memory that worked is not thereby true, and learning requires evaluated experience to change how later situations are processed. One incident is not evaluated experience. Two questions decide it — did anyone approve this principle, and would it hold for a service whose writes are genuinely light?

Both answers are usually no.

Where promotion is wrong

The opposite error deserves equal space, because a chapter that makes promotion look universally beneficial has failed in the same way the essay did.

For this prototype just store it in SQLite.

Promoted faithfully, this becomes a project rule that shared write-heavy services use embedded storage. Every future service inherits a constraint nobody chose, and the system that promotes it looks diligent.

The danger is not only losing experience. Remembering too strongly is also failure.

This is the complement of the Perfect Memory Paradox rather than a repeat of it. Chapter 1 named the cost of keeping everything; this is the cost of promoting everything. A system that escalates a task note into an enforceable invariant has not remembered more accurately. It has remembered more loudly, and the volume is mistaken for the evidence.

The two reports above agree on a mechanism — unasked-for decisions escalating into prohibitions, rules cited that were never approved — and the two scaling accounts point in opposite directions, from a dozen or so principles to a count in the hundreds. Neither observation is a rate. What can be said is narrower: no established scaling law for promotion exists, and any claim about one is extrapolating from a single setup.

Documentation is not exempt from time

Documentation is not exempt from Chapter 8. The sentence

documented → true forever

is false in exactly the way its retraction is obvious to anyone who has worked in a repository.

The canonical history supplies a weaker but honest version of this. The decision record stating that new event-store work should target PostgreSQL is currently correct, and nothing supersedes it — but a derived runbook restates the same guidance to operators without recording that it restates rather than decides. Two documents now assert the same rule with different epistemic standing, and neither says which it is. That is already the failure, before any decision has changed.

The general case is therefore straightforward to state and awkward to demonstrate on this corpus: a durable artifact needs temporal semantics — when it took effect, what superseded it, and when it was last checked against reality. A document that is historically correct and no longer current guidance must be distinguishable from one that is simply wrong, because deleting the first destroys evidence and keeping the second in force corrupts behaviour.

This is also why the staleness manipulation in MEM-DOC-01 proposes a second migration rather than reusing anything already in World A. The corpus as it stands has no superseded decision, so the experiment has to create one. That manipulation is designed, not observed, and nothing in this section should be read as a result.

The external design’s own answer to the problem is a repair pass over stale documents, and it is worth taking seriously precisely because it concedes that prevention is not available and only correction is.

So a durable artifact needs temporal semantics — when it took effect, what superseded it, and when it was last checked against reality. The external design’s own answer, a repair pass over stale documents, is worth taking seriously, because it concedes that prevention is not available and only correction is.

A stale document should not simply be deleted when it remains evidence. Deleting it is how a system loses the ability to explain itself.

Documentation is not enforcement

The chapter’s most important boundary is a negative one.

written rule
≠ discovered rule
≠ selected rule
≠ exposed rule
≠ followed rule
≠ enforced rule

Every link in that chain is a separate system with a separate failure mode, and the essay addresses the first link. The implementation addresses the second, partially, by routing through catalogs. Three separate commenters report the fifth link breaking, and they disagree about which placement would fix it.

Memory owns how experience survives, how it is represented, what authority it carries, and how it becomes available again. What reaches an execution belongs to context. What the agent chooses belongs to reasoning. What is mechanically prevented belongs to the runtime. Evidence of what actually happened belongs to observability. This book stops at the edge of those questions and states them rather than answering them.

One consequence is worth stating because it is easy to miss. An enforcement rule can perfectly prohibit the wrong behaviour while completely missing the intent that motivated it — a specification satisfied, an intent not preserved. That is the Language book’s representation ladder arriving at the end of the Memory book, and it is the strongest available reason not to let this chapter grow into a runtime chapter.

Scope, and what must not be shared

Some memory should never become project documentation at all. Private corrections, personal preferences, relationship context, cross-project habits, temporary intentions, and anything sensitive are poor candidates for a shared artifact regardless of how stable they look.

The external design’s three-partition split is a reasonable answer, and its documented conflict hierarchy — private over user over shared — encodes the right instinct: the narrower scope should win, because the more specific instruction is the one that was actually intended.

But its roadmap for learning durable user observations is unimplemented, so the hardest version of this problem remains open. Personal memory is the case where promotion is most dangerous and least understood, because over-promotion here is not a maintainability problem but a privacy failure.

Does documentation preserve understanding?

The objection that most deserves a hearing is the oldest one.

Naur’s argument is that a program and its documentation do not constitute the programmer’s understanding of a system — that the theory in the programmer’s head is the actual artifact, and the code is its projection. Read literally, that says documentation can never substitute for memory.

It does not transfer cleanly to a system that never had the theory to begin with, and this chapter does not claim it does. Used carefully, it raises a sharper question:

code + documentation + decisions + history
+ rationale + failures + current state
        ↓
reconstruction of the part of the theory this task needs

If that reconstruction is the real operation, then the durable artifact does not need to contain understanding. It needs to contain enough evidence, kept findable and correctly scoped, for understanding to be rebuilt when needed — and no more than the task requires.

This is the same shape as the book’s context argument: memory is durable, context is selected, and the whole theory need not be loaded. The theory is not stored. It is reconstructed from evidence, for this task, on demand.

Book hypothesis. Memory preserves evidence from which understanding can be reconstructed, not understanding itself. Documentation is one representation that makes that evidence easier to find and easier to audit.

The chapter does not claim this is true. It claims it is the sharpest available framing of what documentation is for, and that framing is compatible with both the essay’s case and Naur’s objection.

The evidence ladder, extended

The book’s ladder measures whether retained past reaches behaviour. Promotion needs a parallel branch, because durable knowledge has its own path to influence, and it is not the same path.

experience occurred
    ↓
experience preserved
    ↓
candidate lesson derived
    ↓
scope identified
    ↓
evidence attached
    ↓
promotion authorised
    ↓
durable artifact created
    ↓
artifact remains current
    ↓
artifact discovered
    ↓
artifact reaches context
    ↓
artifact influences behaviour

The first seven rungs are memory’s business. The last four belong to the essay’s, the thread’s, and the context book’s — and none of them is measured by anything in this manuscript.

Note where the essay’s contribution lands. It improves inspectability, editability, versionability, and human auditability — the middle of the artifact’s life. It says nothing about discovery, selection, exposure, or obedience. The essay’s own architecture concedes the first of those by routing freeform documents through an agent-read catalog.

That is why the strongest defensible reading of the whole challenge is narrow:

Book hypothesis. A documentary design trades an invisible ranking failure for a visible staleness failure. The trade favours documentation when the knowledge is project-scoped and human-approvable, because the second failure can be found and corrected by a person and the first cannot.

It is a claim about failure visibility, not about behaviour. It is also the honest answer to the essay’s own concession that its workspace goes stale.

What was measured, and what was not

This section is short because the honest version is short.

No agent has been run. The experiment designed for this chapter — comparing no persistence, raw history, strong retrieval, extracted memory, curated documentation, and a hybrid, over a World A history that mutates after durable knowledge exists — has not been run. Neither has the promotion-boundary experiment. The environment has no model credentials and no inference adapter, and the harness supports scripted policies only.

What was run is narrower and is a result about the instrument, not about any memory system: a check that the promotion scorers separate the failure classes they are meant to separate. A scorer that marks everything wrong would pass a naive accuracy check while measuring nothing, so the tests assert separation instead.

Frozen cases, twelve; reproducibility address promo-instrument-20261004T143000Z.

The oracle reaches the ceiling on every metric, so the scorers are satisfiable. Promoting everything fails authority and scope on every case and flags exactly the four cases whose ground truth is a task note or an episode. Promoting nothing never self-authors authority and misses exactly the three cases carrying a generalising label. The two failure modes separate cleanly.

One asymmetry in that table is worth carrying forward, because it is a property of the metric rather than of any system:

A system that promotes nothing scores 1.00 on licence_correct, because forgetting never grants itself authority. Read alone, the metric would recommend a system that remembered nothing.

That is why licence is never reported without missed promotion beside it. It is a small demonstration of the book’s standing refusal of a composite score, and a composite would have destroyed it.

None of this is evidence about agents, documentation, or promotion. It is evidence that the instrument can tell the failure modes apart when a reader eventually supplies them.

Book result (instrument only, scripted policies, twelve frozen cases, no model inference). The promotion scorers separate over-promotion from under-promotion in the predicted directions, and the oracle reaches the ceiling on all seven metrics. No claim is made about any memory system’s promotion behaviour.

Verdict. The promotion mechanism enters the architecture as EXPERIMENTAL / DEFERRED, alongside Chapter 15’s growth policies and Chapter 16’s learning boundary. The failure classes are measurable and the instrument can separate them. The mechanism is not earned, because no failure has yet been demonstrated in a system that was given the chance to fail.

What remains unsolved. Whether a promoted artifact reaches behaviour more often than an unpromoted one; whether catalog-mediated discovery beats similarity ranking on the same corpus; how often promotion is correct; whether personal-scope knowledge belongs in project documentation at all; and whether Naur’s reconstruction reading survives contact with a working system.

Research foundations

The essay is the chapter’s trigger and is treated throughout as an author argument rather than as an established result.

Kevin Liao’s Agents Don’t Need Memory. They Need Documentation (3 October 2026, personal essay, read in full) supplies the five-part critique and the consult-build-update alternative. It contains no measurement, and its companion comparison document asserts that all memory plugins share one architecture without a source.

Operator Memory (BSD-3-Clause, read in full at the repository on 4 October 2026) supplies the case study: three partitions with a documented conflict hierarchy, deterministic session orientation, the catalog-versus-index split, an explicit staleness concession, and agent-authored documentation. It is vendor-reported and not line-audited. Roadmap items are not shipped features.

The Hacker News discussion (230 points, 130 comments, read in full) supplies practitioner reports on agent-authored authority, instruction survival across placements, and token cost, plus the recorded disagreements. It is community discussion and contains no controlled comparison. It is never used as empirical evidence.

Michael Nygard’s Documenting Architecture Decisions (Cognitect, 2011) supplies the pre-AI precedent: short durable records, one significant decision each, explicit status and supersession, kept in the repository. This is established engineering practice, independent of AI, and predates the whole question by fifteen years.

Peter Naur’s Programming as Theory Building (1985, peer-reviewed) supplies the counterweight. It is used for the epistemic problem only, not as evidence about AI cognition.

References

• Kevin Liao, Agents Don’t Need Memory. They Need Documentation (3 October 2026, personal essay).
• Kevin Liao and aerovato, Operator Memory: the self-improving context engine for coding agents (BSD-3-Clause, vendor-reported; README and architecture documents).
• Hacker News discussion of the above, thread 49945933 (community discussion; not empirical evidence).
• Michael Nygard, Documenting Architecture Decisions (Cognitect Blog, 15 November 2011).
• Peter Naur, Programming as theory building (Microprocessing and Microprogramming, 1985).