Part 2 — Get the Model Out of the Chat Box

Nothing went wrong

The task was small — repair one paragraph:

The new cache makes every page load 73% faster, according to the platform team [S1].
It stores rendered fragments close to readers.

It had three written criteria: the paragraph must contain no percentage figure, the source marker [S1] must appear exactly once, and the second sentence must survive word for word.

The call went through the recorded path from Chapter 11 and came back with this:

The new cache is intended to make page loads faster, according to the platform team [S1].
It stores rendered fragments close to readers.

The answer is right, with the dialect handled and generation finished normally, and the attempt was interpreted and preserved. A new process reopened the ledger and asked about the task:

call_status:        succeeded
generation_state:   complete
task_completion:    incomplete   (no acceptance recorded)

Nothing went wrong. The task is still not complete.

That is the line Chapter 11 ended on, task_status: not automatically completed, and every chapter since has carried it forward. Chapter 1 called HTTP 200 a transport fact, not a work outcome. This chapter makes that distinction executable.

What facts must exist before work can be called complete?

Four facts that are not the same fact

Most systems collapse four different events into one status field:

generation succeeded
        ≠
artifact checked
        ≠
artifact accepted
        ≠
task completed

Each is owned by something different:

Task
 │
 ├── Call ─────────── generated artifact          (what the model produced)
 │
 ├── Check ────────── evidence about that artifact (what a deterministic procedure observed)
 │
 ├── Acceptance ───── cites the call, the artifact and the check,
 │                    made by an eligible role     (a decision)
 │
 └── Completion ───── valid only when caused by that acceptance

The output belongs to the model call. “Done” belongs to the task. Between them lies the process — the model produces cognition inside a task that the runtime still owns:

    flowchart TD
    subgraph TASK["task — owned by the runtime, not the model"]
        direction TB
        C(["model call<br/><i>produces cognition</i>"]) --> AR["artifact<br/><i>what the model produced</i>"]
        CK["check<br/><i>deterministic observation</i>"] --> EV["evidence<br/><i>about that artifact</i>"]
        AR --> AC["acceptance<br/><i>cites call + artifact + check</i>"]
        EV --> AC
        AC --> CO["completion<br/><i>valid only when caused by acceptance</i>"]
    end
  

The output belongs to the model call. “Done” belongs to the task. Between them lies the process.

The title of this chapter is not a complaint about models. A model is very good at producing the first line of that diagram. It has no access to the other three. It cannot know whether its bytes were checked, which bytes the checker saw, whether anyone with standing agreed, or whether that agreement was recorded. Those are facts about the process around the call, and only the process can hold them.

The check belongs at the end

In 1984, Saltzer, Reed and Clark worked through a problem they called careful file transfer (Saltzer, Reed and Clark, 1984). A file has to move from one computer’s disk to another’s without damage. Many things can fail along the way. The disk can return bad data. The software can make a buffering mistake. Memory can flip a bit. The network can drop, alter or duplicate packets. A host can crash partway.

You could make every layer more reliable. Their point was that this cannot finish the job. The transfer is trustworthy only when the application at the far end reads back what was actually stored, compares a checksum with the original, and commits only if they agree. A reliable network removes one threat and leaves the rest. In their words, the end-to-end check of the file transfer application must still be implemented, however reliable the lower layers become.

They also told a “too-real” story. A network at MIT checksummed every hop between gateways. One gateway had a fault that occasionally swapped a pair of bytes while copying between buffers, where no checksum covered the data. Source files passing through it were corrupted, and their owners were left with the ultimate end-to-end check: comparing against old listings by hand.

This chapter applies that argument to a model call. The mapping is ours, not the paper’s:

• Transport success, a parsed body, a normal finish reason and a clean interpretation are the per-hop checksums. They are worth having. Chapter 11’s retry decisions and Chapter 12’s truncation detection depend on them.
• None of them is the end-to-end check. The end is the task. The check has to examine the artifact that will be used, against the task’s own criteria, and the commit decision has to follow from that.

Saltzer and colleagues treated lower-level reliability as useful for performance rather than sufficient for correctness. Carried over, a succeeded call stands to a completed task the way a reliable network stands to a correctly stored file.

Which bytes, from where

The end-to-end argument says where the check must happen. Knowing that the checked thing is the thing you are accepting takes a separate provenance rule.

The W3C PROV data model supplies the vocabulary (Moreau and Missier, 2013):

• An entity is a thing with some fixed aspects.
• Generation is the completion of production of a new entity by an activity.
• Derivation turns one entity into another.
• Association assigns an agent responsibility for an activity.
• Attribution ascribes an entity to an agent.

Put the paragraph repair into those terms. The call and its final attempt are the generating activity. The artifact is an entity, and its fixed aspect is its SHA-256. A check is an activity that used that entity. An edited paragraph is not the same entity with a small change. In PROV’s terms it is a different entity derived from the first. PROV supplies that distinction. What a check on one entity means for another is decided by the rule built on top of it here, which is CodeAI’s: nothing established about the first entity transfers to the second.

This is why an acceptance in this chapter does not say “the paragraph is fine”. It names an exact chain:

task → call → final attempt → interpretation → artifact sha256 → check(s) → acceptor role

The rule

CodeAI Stage 14 adds one operation and one projection, task-acceptance-v1. An acceptance is appended as task.accepted only if every reference validates against the ledger:

• Authority for the acceptor grants a new capability, accept, separate from the capabilities that produce work.
• The acceptor must carry a different label from the actor that produced the output.
• The task must exist, with the acceptance citing the hash of its declared criteria. You accept against the criteria that were set, not against criteria you prefer.
• The source call belongs to this task, and its recorded status is succeeded.
• The cited attempt is the call’s final attempt.
• The cited interpretation is the one the call’s status decision rested on, and it records complete generation.
• The artifact’s SHA-256 equals the output preserved in that attempt’s envelope, and the stored bytes are intact.
• At least one check is cited. Each was recorded for this task, targeted artifact:sha256:<those bytes>, and completed PASS.

In the experiment’s producer, the check and the acceptance are two ordinary calls. Reduced from the executed code:

check = runtime.run_check(CheckRequest(
    check_id=check_id, task_id=task_id,
    command=(sys.executable, "paragraph_criteria_check.py",
             "--path", artifact_path, "--sha256", sha),
    target=artifact_target(sha)),              # "artifact:sha256:<those bytes>"
    verifier=LocalCommandVerifier())

completion = runtime.accept_task(AcceptanceRequest(
    acceptance_id=str(uuid4()), task_id=task_id, actor_id="reviewer",
    criteria_sha256=criteria_sha256(CRITERIA),
    artifact_sha256=sha,
    source_call_id=call_id, source_attempt_id=attempt_id,
    source_interpretation_id=interpretation_id,
    check_ids=(check_id,)),
    authority=Authority(frozenset({Capability.ACCEPT})))

Every field in the acceptance is an identity the runtime can look up. Nothing in it says the paragraph is good; that judgment lives in the check it cites.

If all of that holds, the runtime appends task.accepted and then task.completed, whose causation_id points at the acceptance. If anything fails, it appends task.acceptance_rejected with the reasons and changes nothing else.

Completion is then not a field anyone sets. It is derived:

completed(task) :=
    exists task.accepted A for the task
    and exists task.completed C
    and C was caused by A

A can only have been appended by the validating operation, and A carries its references. The ledger contains facts; the projection derives state. A task.completed event on its own is not authoritative. It is a claim, and the projection asks what acceptance it came from.

Experiment 1: generation is not completion

The demonstration runs offline. The model response is scripted and sent through the real recorded path: the OpenCode adapter, the Chat Completions codec, and the attempt and interpretation machinery from Chapters 11 and 12. That is deliberate. The question here is not whether a model can repair a paragraph. It is what the process may conclude once one has.

Every step runs as a separate operating-system process over one SQLite ledger:

1. produce creates the task, makes the call, stores the output as an artifact, and writes a checkpoint file only after the ledger has committed the ten events of the task and its call.
2. The parent waits for the checkpoint. Then the run splits in two:
   • clean exit: the child returns normally, exit code 0.
   • abrupt termination: the child is still running and the parent kills it (TerminateProcess, no cleanup), exit code 1.
3. inspect is a new process that reopens the ledger and projects the task. It has no transport function to call.

Both runs reopened identically:

Reading the process’s state changed nothing and asked the model nothing. Restarting invented no acceptance, and it did not re-run the cognition to be safe. The call’s manifest, start, observation and completion each appear exactly once before and after.

The two interruptions are labeled separately on purpose. The killed process died at a durable checkpoint, after the ledger had committed. That shows the facts survive a process that never shuts down cleanly. It stops short of showing that every crash point is safe, and this chapter makes no such claim.

Experiment 2: evidence, then a decision

A third process verifies and accepts. It runs paragraph_criteria_check.py as its own subprocess, and that script imports nothing from CodeAI. It reads the artifact file, recomputes its SHA-256, refuses the file if the hash differs from the one it was given, and evaluates the three criteria:

artifact_sha256_observed  89de30226cb4b905…  bytes_match: true
no-percentage   passed
marker-once     passed
sentence-kept   passed

A passing check is still not completion. In a separate ledger, one negative case ran the same call and the same passing check and stopped there. It projects incomplete, basis no acceptance recorded. Evidence about an artifact is not a decision about a task.

Then the third process accepts, as a role labeled reviewer holding only accept:

+ task.accepted    actor=reviewer  authority_basis=[accept]  check_ids=[check-293e…]
+ task.completed   actor=runtime   caused_by=task.accepted

Two events. Then it does two things a real system does by accident, and a fourth process looks at the result:

• Sends the identical acceptance again, with a new request id: 0 events. The projection returned is the same.
• Runs a second passing check and submits an acceptance citing both checks: refused, conflicting_acceptance. One rejection event is recorded, and there is still exactly one completion.
• A fourth process reopens: completed, 17 events. The acceptance names the same artifact hash and the same call. The interpretation and the original decisions are unchanged from before acceptance.

The repeat is the part distributed-systems people will recognize. Helland’s position paper on building without distributed transactions starts from the assumption that messages arrive at least once. A recipient must be designed to ignore redundant messages. It typically does that by remembering what it has already processed and answering a repeat the way it answered the original (Helland, 2007). CodeAI applies that reasoning to acceptance. What it remembers is the acceptance’s identity: everything about it except the request id. A reviewer who clicks twice, or a script that retries after a timeout, gets the decision already on record, not a second one.

How CodeAI handles the gap between the two appends is its own design, tested in this experiment rather than taken from the paper. task.accepted and task.completed are two writes, and this runtime does not pretend they are one. A negative case injected a failure after the acceptance was written and before the completion. On reopen, the task projects incomplete, with acceptance_pending_completion: true. Sending the identical acceptance again appended the one missing task.completed and nothing else.

That is a recoverable protocol, not an exactly-once transaction. The failure was an injected exception, not a killed process, and the ledger has one writer. Chapter 16 takes up restarting versus repeating properly.

Experiment 3: try to cheat

Each case below ran in its own ledger. Apart from the last row, none produced an acceptance, and none projects as completed:

Three of these carry most of the argument.

A succeeded call is incomplete. No failure is needed to show this: the call succeeded and the work was not done. Any pipeline that turns a model’s success into a task’s success has merged two facts the ledger keeps apart.

The edited artifact passed its check and was still rejected. The edit was small: “is intended to make” became “should make”. The criteria check passed on the edited bytes, and that check was perfectly valid. It was also about a different entity from the one the call generated. Verification of some bytes is not evidence about different bytes. Chains break exactly here in practice: someone tidies the output after review, or a formatter runs after the tests, and a green check is attached to something it never saw.

A bare task.completed does not complete anything. Anyone with write access to a status field can set it to done. Here someone appended the completion event directly, and the projection asked what acceptance caused it. There was no answer, so the task stayed incomplete. The ledger is not replaying commands. It interprets evidence according to a protocol.

The truncated case deserves a line too. The text happened to pass the criteria. The generation still did not finish, and the runtime refused to accept output from an unfinished generation just because what arrived looks acceptable.

Finally, a verifier that shares no code with CodeAI opened every ledger read-only, together with the stored bytes, and checked the chain itself:

• one acceptance and one completion in each run, the completion caused by and sequenced after the acceptance
• the accepted hash equal to the candidate bytes, which equal the text in the stored transport body
• every cited check linked, recorded for the task, aimed at those bytes, passing, and earlier than the acceptance
• history before acceptance an unchanged prefix of history after
• across the negative ledgers, exactly one causal acceptance–completion pair: the repaired interruption

It reached the same answer as Runtime.task_completion().

Completion is not something the model reports. It is something the process can prove about its own history.

What building it exposed

The first time something depends on an identity, you find out whether the identity was recorded.

Acceptance needed to find a call’s completion record by call id. For calls made through the gateway adapter, it could not. The adapter’s send() returns a result with an empty call_id, and the recorded path copied that into call.completed. The Stage 12 live ledgers from Chapter 12 contain exactly that: completion events with an empty stream id. Reconstruction had been quietly working around it by deriving status from the last attempt. Nothing had failed, because nothing had yet needed that event’s identity.

The repair went into CodeAI as its own commit (34c16b8), with a regression test that fails against the previous runtime, before the acceptance feature (fe0797d). The Stage 12 ledgers were left as they are. They are a true record of what that code did. Acceptance validation deliberately avoids relying on call.completed at all. It uses the status decision, the attempt records and the preserved envelope, so it works on ledgers written before the fix.

Roles, not yet authority

Be precise about what the accept capability and the self-acceptance refusal establish.

The runtime now represents producer and acceptor as distinct roles, and it refuses to let one actor label fill both. That is structure, and it is useful.

It is not security:

• A label is a string, not an authenticated identity.
• Ledger events are unsigned. Anyone who can write the SQLite file directly could append a matching task.accepted and task.completed pair, and the projection would believe it. The ledger is the trust boundary.
• The criteria check was written by the same author as the task, runs on the same machine, and checks mechanical criteria. It is a deterministic procedure, not an independent judge of whether the paragraph is true. Chapter 21 splits that worry into three questions — is a check independent of what it evaluates, adequate to the property that matters, and bound to the exact state being accepted? — and this check answers them unevenly: bound to the bytes by hash, adequate only to three mechanical rules, independent of the model but not of the task’s author.

Who is actually permitted to make a consequential decision is Chapter 20’s subject. What makes evidence independent of the thing it evaluates is Chapter 21’s. This chapter needed only one step: that “done” is a decision with recorded inputs, not something a component announces.

Where it is still weak

1. Labels are not identities. reviewer and repairer are strings chosen by the caller.
2. The ledger is unsigned. A forged acceptance–completion pair would project as complete.
3. Single writer only. Nothing prevents two processes accepting at the same moment.
4. Crash coverage is narrow. The kill happened at a durable checkpoint, and the window between the two appends was exercised by an injected exception, not a kill.
5. The check is mechanical. It shows the bytes meet three written rules. It says nothing about whether the claim is now true, or whether the rules were the right ones.
6. The model output was scripted. The recorded path is real. No live model was asked, so nothing here measures repair quality.
7. Policy v1 accepts only the call’s own output. A human-edited paragraph cannot be accepted under it. The next version would need to record the derivation and who made it.
8. One task, one artifact, one kind of check. There is no multi-artifact completion, no partial completion, no run status (list_runs still reports active), and no CLI surface.

Do this now

Forty minutes. Find out what “done” means in your system.

1. Take the last ten items your team, pipeline or agent marked done. For each one, write down the exact version of the artifact, the check that ran against those bytes, and who decided. Count the cells you cannot fill.
2. Search for the point where a model call’s success, an HTTP 200 or a green CI run becomes a status called done, complete or resolved. Note whether anything records what that status was based on.
3. In a disposable environment, change the artifact after its check passes. Does anything notice?
4. Set the done status directly, without doing the work. Does anything ask why?

If you are building with an assistant:

Add explicit task acceptance on top of an append-only event log.
A successful model call or a passing check must never complete a task.
- Acceptance names: task id, hash of the task's declared criteria, exact
  artifact sha256, source call and final attempt, the interpretation the
  call's status decision used, and one or more check ids.
- Validate against the log before recording anything: same task; call
  succeeded; generation complete; artifact equals the call's preserved
  output and is intact; each check was for this task, targeted these bytes
  and passed; the acceptor holds an accept permission and is not the
  producer.
- On success append "accepted", then "completed" with a causal link to it.
  On failure append "acceptance rejected" with reasons and nothing else.
- Derive completion only from a completed event caused by an acceptance.
  A completed event with no cause does not count.
- Identical repeats append nothing and repair a missing completion;
  a different acceptance for an accepted task is refused.
- Test in separate processes: produce, kill, reopen (still incomplete, no
  model call), check, accept, reopen (complete). Then try each negative case.
- State the limits: labels are not authentication; single writer; no
  exactly-once guarantee.

Failure modes

• Call success as task success. The call succeeded; the work had not been decided.
• A passing check as completion. Evidence about an artifact is not a decision about a task.
• Checking different bytes than you accept. The edited paragraph passed its check.
• A status field anyone can set. Completion without a recorded cause is a claim, not a fact.
• Letting the producer accept its own output. Even as labels, keep the roles apart.
• Accepting against criteria you changed afterwards. Bind acceptance to the criteria as declared.
• Non-idempotent acceptance. A double click produces two completions.
• Pretending two writes are one. Make the gap recoverable and visible.
• Accepting truncated output because it looks fine. Unfinished generation is not a finished artifact.
• Calling a label an identity. Roles are structure, not security.

What this chapter established

• A successful, correctly interpreted, fully preserved model call left the task incomplete, and that was correct. Generation succeeded ≠ artifact checked ≠ artifact accepted ≠ task completed.
• The end-to-end argument puts the correctness check at the task, against the artifact that will be used. Call and transport success are per-hop reliability.
• Provenance makes the check meaningful: an acceptance cites the generating call, the exact artifact hash, the checks on those bytes, and an eligible role. An edited artifact is a different entity.
• Built in CodeAI as task-acceptance-v1: task.accepted only after validation, task.completed caused by it, refusals recorded, completion derived rather than set.
• In separate processes, with both a clean exit and a killed producer, reopening showed a succeeded call and an incomplete task. Inspecting appended 0 events and made 0 model calls. Check plus acceptance appended 2 events and completed the task. An identical repeat appended 0, a conflicting acceptance was refused, and history before acceptance was unchanged.
• Twelve negative cases, including an edited artifact with its own passing check and a hand-appended task.completed, did not complete. A verifier sharing no code with CodeAI reached the same conclusions from the raw ledgers.
• The interruption between acceptance and completion is recoverable, not atomic.
• Roles are represented and kept apart. Authentication, signing, concurrency and independent verification are not established.
• Building it exposed a real defect: gateway calls recorded completion without a call id. It was fixed in its own commit, and the old ledgers were left as evidence.

Next

Part 2 took the model out of the chat box one boundary at a time. Chapter 11 recorded the call. Chapter 12 survived several dialects. Chapter 13 made the numbers honest. This chapter put the call inside a process that can say, from its own history, whether the work is done.

Look again at what the acceptance cites: a call, an attempt, an interpretation, an artifact, checks. It does not cite what the model was shown. The check establishes that the output meets the criteria. Nothing in the chain establishes that the input was the right input, or lets another process reconstruct it. Part 3 starts there.

Continue with Context Is an Input, Not a Transcript.

References

• J. H. Saltzer, D. P. Reed, and D. D. Clark. End-to-End Arguments in System Design. ACM Transactions on Computer Systems, vol. 2, no. 4 (November 1984), pp. 277–288. https://web.mit.edu/Saltzer/www/publications/endtoend/endtoend.pdf
• Pat Helland. Life beyond Distributed Transactions: an Apostate’s Opinion. 3rd Biennial Conference on Innovative Data Systems Research (CIDR), 2007. https://www.cidrdb.org/cidr2007/papers/cidr07p15.pdf
• Luc Moreau and Paolo Missier (eds.). PROV-DM: The PROV Data Model. W3C Recommendation, 30 April 2013. https://www.w3.org/TR/prov-dm/

Implementation sources: CodeAI 34c16b8 (call identity repair: Runtime._interpret_attempt; tests/test_opencode_gateway.py::test_recorded_call_completed_carries_logical_call_id) and fe0797d (src/codeai/acceptance.py: task-acceptance-v1, AcceptanceRequest, TaskCompletion, accept_task, project_task_completion; Capability.ACCEPT in src/codeai/domain.py; Runtime.accept_task and Runtime.task_completion; tests/test_task_acceptance.py; experiments/task_completion_demo.py; experiments/paragraph_criteria_check.py). Suite 289 passed. Evidence: experiments/applied-ai/evidence/task-completion/offline-fe0797d/, run from a clean tree at fe0797d (manifest, both process chains with checkpoints, termination records, before/after projections and event exports, check reports, acceptance chains, negatives/, hashes.json), and independent-verification/ (a verifier sharing no code with CodeAI, and its result). Stage 12 ledgers showing the empty call id: experiments/applied-ai/evidence/protocol-conformance/live/*/events.json.