Part VII — Debugging Agents

The fork worked — so the forked step caused it? Not yet

Chapter 40’s fork F1 (HTML instead of PDF at step 12) succeeded in 3/3 trials. The practitioner is tempted to close the case: the PDF action caused the failure. But the forward runs also used a different source snapshot, ran at a different hour, and consumed a parse path with its own cache — any of which could carry the change. Sensitivity shows that the downstream moved; it does not show what moved it.

OBSERVATION: fork F1 differs from the original in the intended action AND in two uncontrolled companions: source snapshot ID (s-881 → s-884, live-data waiver active) and parse-cache state (cold → warm). HYPOTHESIS H1 (action-causal): the step-12 action change alone produces the predicted downstream signature. H2 (companion-causal): a snapshot or cache difference, not the action, carries the change. H3 (joint/unknown): the effect needs the combination, or the trials cannot separate the carriers. INFERENCE: none yet — H1/H2/H3 separate only by predicted-vs-observed comparisons under isolated intervention, never by fork success alone.

This chapter’s question: when does a replay earn a causal verdict — and what must you predict before running to deserve one?

Why “it worked when I changed it” fails first

The obvious move — treating fork success as proof — fails because interventions travel with companions. Four confusions dominate:

1. Companion smuggling. Live snapshots, warm caches, and clock-dependent tool behavior ride along with the intended change. The fork tests a bundle, not a cause.
2. Prediction-free reading. Declaring victory from whatever improved downstream. Without a pre-written predicted signature, any difference confirms any story.
3. Asymmetric comparison. Scrutinizing the original’s flaws while grading the fork on prose quality. Both sides get the same deciding signatures or neither is read.
4. Single-bundle generalization. “PDF fetching breaks agents” from one pinned run. A causal verdict covers the tested bundle under the tested pins — scope creep is not included.

OPINION: an unpredicted intervention is a lottery ticket stapled to a lab report — a win proves luck, not mechanism. Predict the downstream, then run.

The mental model: causal replay is an experiment with three arms — original, null replay, and isolated intervention — judged by predicted-vs-observed divergence. The diagnosis predicts a specific downstream signature change (which steps, which hashes, which outcome, by when); the replay observes it; the verdict compares the two under companion control (freeze or explicitly vary each companion). Matches under isolation support H1 for the bundle; mismatches or companion-driven matches force H2/H3.

This is Causal Testing (Chapter 1) applied to a trajectory rather than a program value — select executions by counterfactual causality, intervene, observe (Johnson, Brun & Meliou, 2020). The interior-model version of the same experiment is activation patching — a form of causal mediation analysis introduced to language models by Vig et al. (2020), the family that also includes ROME’s causal tracing of factual associations (Meng et al., 2022) and distributed alignment search (Geiger et al., 2024). All of them force one internal variable and measure the downstream, the interior twin of this chapter’s forced fork — and they have exactly the pitfalls this chapter guards against. Zhang and Nanda showed that the choice of evaluation metric and of how you corrupt the baseline can flip the conclusion, so the metric and the companion set have to be fixed in writing before the run, not chosen after seeing the result (Zhang & Nanda, 2024). More recently, re-deriving the patching estimand from causal mediation theory shows that the “isolated” effect through one component still carries interaction terms with the state of other components (Vaidyanathan et al., 2026) — the interior version of this chapter’s companion confounding, and the reason even an isolated arm needs its swap checks.

The method: predict, isolate, compare divergence

Run the causal test in this order — prediction first, always:

1. Pre-write the predicted divergence. Before any modified run: which downstream steps change, to what values (outcome codes, state hashes, novelty ratios), and which stay fixed. Vague predictions (“better results”) are inadmissible.
2. Freeze companions. Snapshot IDs, cache states, clocks, and external data pinned or explicitly reset per trial; every remaining companion listed as a live-data waiver with its legitimacy argument. Unlisted drift → H3, not H1.
3. Run three arms. Original trace (frozen), null forks ×3 (replay validity), isolated-intervention forks ×3 (one change, companions frozen). Component-freeze variants (hold the suspected component’s outputs fixed while varying the rest) where the tool supports them.
4. Compare predicted vs. observed. Score the match per trial on deciding signatures only: full match (all predicted fields), partial (some), mismatch (predicted change absent or unlisted change dominates). Judge both arms symmetrically.
5. Scope the verdict. Causal support, if earned, names the bundle, pins, and fork point — “step-12 PDF action causal for run R under pins P” — with companions listed. No task-family or model-generality language.

    flowchart TD
    PR["pre-write the predicted divergence: which steps change, to what values, which stay fixed"] --> FC["freeze companions: snapshot IDs, caches, clocks — or list each as a waiver with its argument"]
    FC --> AR["run three arms x3: original (frozen), null replay, isolated intervention (one change)"]
    AR --> SW["companion-swap check: the intervention + ORIGINAL companions"]
    SW --> Q{"predicted-vs-observed, scored symmetrically on deciding fields"}
    Q -->|"isolated matches prediction; swap still reproduces the effect"| H1["H1 action-causal FOR THIS BUNDLE under these pins"]
    Q -->|"swap with the original action reproduces the effect"| H2["H2 companion-causal — repair the snapshot / cache, not the step"]
    Q -->|"partial match, or companion-dependent"| H3["H3 joint / unknown — narrower forks or explicit UNKNOWN, companions listed"]
  

CAUSAL SCORECARD (constructed; intervention: step-12 PDF->HTML, pins P):
predicted: parse_ok by step 14 (code 0, novelty >0.5 steps 12-20,
  terminal state h9f2-class) | companions frozen: snapshot s-881, cache cold
null ×3:     parse_failed ×10, novelty 0.0, terminal h7a1 (matches original)
isolated ×3: parse_ok@14/14/15, novelty 0.8/0.7/0.8, terminal h9f2-class ×3
companion-swap check (HTML action + ORIGINAL s-881/cold): same signature ×2/2
RULE: prediction matched on deciding fields with companions frozen ->
H1 supported FOR THIS BUNDLE. The swap check (not the headline fork)
does the causal work.

OBSERVATION (constructed illustration, not a measured run): isolated forks match the pre-written signature on all deciding fields across trials; the companion-swap check reproduces the effect with original snapshots, exonerating the snapshot companion for this instance. UPDATED BELIEF: H1 supported for run R under pins P at step 12; H2 exonerated here for the snapshot/cache companions tested; H3 live for untested companions (clock, network path) — listed, not ignored.

Example: the component freeze that exonerates the model

A competing story blames the model (“weak parser-following”). The practitioner freezes the component instead of swapping models — hold the parser outputs fixed, vary the fetch action:

# causal replay: predicted divergence, isolated arms (no verdict yet)
prediction = {"parse_ok_by": 16, "novelty_floor": 0.5, "terminal_class": "h9f2"}
companions = freeze_all(snapshot="s-881", cache="cold", clock="t-fixed")  # OBSERVATION
nulls = [replay(bundle, 12, None) for _ in range(3)]  # validity arm
iso = [replay(bundle, 12, "fetch_html") for _ in range(3)]  # isolated arm
swap = [replay(bundle, 12, "fetch_html", companions=ORIGINAL) for _ in range(2)]
# component-freeze variant: parser outputs held to recorded bytes while the
# fetch action varies -> if effect persists, parser-component cause exonerated
# for this bundle. Score predicted-vs-observed per trial, symmetrically.

In the constructed case the freeze variant sustains the effect: identical parser bytes, different fetch action, predicted signature still matched. The model/parser-blame story is exonerated for this bundle by experiment — not by argument, and not beyond these pins. The licensed sentence is deliberately small, and that smallness is the point: causal replay mints narrow, tradable verdicts instead of sweeping ones.

No confidence movement, no judge preferring the HTML trajectory, no agreement across the three trials treated as bonus proof, and no downstream symptom (“users preferred the HTML answers”) scores the card. Predicted fields match observed fields under isolation, or H1 is not earned.

Research lineage: reject confidently, confirm cautiously

The strongest honest sentence is a rejection. Redwood Research’s causal scrubbing formalizes a hypothesis about a mechanism as a claim that certain things can be resampled without changing behavior, and tests it by doing exactly that — and its authors are explicit that the method can reject a hypothesis but never confirm one (Chan et al., 2022). Causal scrubbing has its own documented limits — it can pass a hypothesis whose resampling distribution is too permissive — so “reject confidently” is the robust half of the asymmetry. Causal replay inherits it. The companion-swap check that shows a snapshot difference carries the effect is a clean rejection of the action-causal hypothesis; the isolated fork that matches the prediction is support for it, scoped to the bundle, not proof. “H1 supported for run R under pins P” is deliberately the weakest verdict that the evidence licenses.

Intervention is not observation, and the difference is the whole point. The distinction the chapter runs on — that watching a fork succeed (observation) is not the same as forcing one variable and measuring the effect (intervention) — is Pearl’s, and it is why the three-arm design with frozen companions is not optional rigor but the definition of a causal test.

Lab 41: predicted-divergence test with companion control (proposed)

PROPOSED, not executed: no author-measured results are reported. The evidence this chapter requires is the reader’s own scorecard.

Setup. Take the Chapter 40 replay bundle with validated nulls and one candidate cause at the fork point. Freeze or explicitly waive every companion (snapshots, caches, clocks). The intervention isolation (bundled fork vs. isolated fork with companions frozen) is the independent variable; fork point, pins, and prediction are controlled.

Task.

1. Before running, write H1/H2/H3 with distinct predicted-vs-observed patterns: H1: “isolated fork matches prediction on all deciding fields”; H2: “companion-swap (original action + new companions) reproduces the effect”; H3: “partial match or companion-dependent effect.”
2. Run null ×3, isolated ×3, and at least one companion-swap ×2; score each trial full/partial/mismatch against the prediction.
3. Write the scoped verdict sentence with bundle, pins, fork point, and companion list.

Success criterion. A scored card with pre-written predictions, companion list with freeze/waiver status, and a scope-named verdict. A successful-but-unpredicted fork is explicitly not completion.

Companion tool: Component Freeze/Re-run Tool

What it accepts: the replay bundle with validated nulls, the pre-written predicted divergence, the companion inventory (frozen vs. waived with arguments), and the single intervention. What it performs: it freezes or varies named components per trial (hold parser bytes, swap snapshots, reset caches), executes null/isolated/swap arms with contracted forward traces, scores predicted-vs-observed per trial on deciding fields, and refuses causal language on unpredicted or companion-confounded effects. What it can establish: whether the predicted divergence occurred under isolation, which companions are exonerated by swap checks, and what scoped causal sentence the trials support — for the tested bundle and pins only. What it cannot establish: generality across tasks, models, or revisions; untested-companion innocence; or future reliability. It never treats narration, confidence, scores, agreement, single-arm success, or downstream symptoms as causal evidence. How its output changes your next action: H1-scored → single-variable repair at the convicted step with its own predicted outcome; H2 → repair the companion (snapshot/cache policy), not the step; H3 → narrower forks or explicit UNKNOWN with companions listed — never a broad causal claim.

Paper form, sufficient for this chapter:

Bundle/pins: ___  Fork: step ___  Prediction: ___
Null ×3: ___  Isolated ×3: ___ (match: full/partial/mismatch per trial)
Swap ×2: ___  Companions: frozen ___ waived ___
VERDICT (scoped): ___  NEXT REPAIR: step / companion / narrower fork

Where a software implementation does not yet exist in the reader’s stack, this record is the tool. Prediction before running, scope after.

Reusable procedure: earn every causal sentence

1. Predict — downstream fields, values, and fixed companions, in writing, before running.
2. Freeze — companions pinned or waived with arguments; no unlisted drift.
3. Three arms — original, null ×3, isolated ×3, plus swap checks for live companions.
4. Score symmetrically — predicted-vs-observed per trial on deciding fields only.
5. Scope — bundle, pins, fork point, companions in the verdict sentence; nothing beyond.

Failure modes

• Fork-equals-cause. Reading Chapter 40 sensitivity as causation. Sensitivity is the candidacy; this chapter is the election.
• Vague prediction. “Should improve.” Unfalsifiable predictions confirm everything and prove nothing.
• Smuggled companions. Live snapshots or warm caches riding with the intervention. Unlisted drift forces H3.
• Asymmetric grading. Prose-judging the fork while hash-judging the original. Same signatures both sides.
• Scope creep. “PDF fetching is fixed.” The verdict names one bundle under one pin set — audible narrowness required.
• Score-based causation. Citing confidence gains or judge preference as the mechanism. Predicted fields match or H1 waits.

Limits, per contract: one scorecard covers one bundle, one fork point, one pin set; untested companions stay live suspects; causal support is bundle-scoped and revision-bound. UNKNOWN wherever predictions were unwritten, companions unfrozen, or nulls unvalidated.

References

• Brittany Johnson, Yuriy Brun, and Alexandra Meliou. Causal Testing: Understanding Defects’ Root Causes. Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering (ICSE), 2020. https://doi.org/10.1145/3377811.3380377
• Fred Zhang and Neel Nanda. Towards Best Practices of Activation Patching in Language Models: Metrics and Methods. International Conference on Learning Representations (ICLR), 2024. https://arxiv.org/abs/2309.16042
• Jesse Vig, Sebastian Gehrmann, Yonatan Belinkov, Sharon Qian, Daniel Nevo, Yaron Singer, and Stuart Shieber. Investigating Gender Bias in Language Models Using Causal Mediation Analysis. Advances in Neural Information Processing Systems 33 (NeurIPS), 2020. https://proceedings.neurips.cc/paper/2020/hash/92650b2e92217715fe312e6fa7b90d82-Abstract.html
• Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and Editing Factual Associations in GPT. Advances in Neural Information Processing Systems 35 (NeurIPS), 2022 (arXiv:2202.05262). https://arxiv.org/abs/2202.05262
• Atticus Geiger, Zhengxuan Wu, Christopher Potts, Thomas Icard, and Noah D. Goodman. Finding Alignments Between Interpretable Causal Variables and Distributed Neural Representations. Proceedings of the 3rd Conference on Causal Learning and Reasoning (CLeaR), 2024 (arXiv:2303.02536). https://arxiv.org/abs/2303.02536
• Sankaran Vaidyanathan, David Arbour, Aaron Mueller, et al. The Curse of Multiple Mediators: Hidden Interaction Effects in Activation Patching. arXiv:2606.27510, 2026. https://arxiv.org/abs/2606.27510
• Lawrence Chan, Adrià Garriga-Alonso, Nicholas Goldowsky-Dill, Ryan Greenblatt, Jenny Nitishinskaya, Ansh Radhakrishnan, Buck Shlegeris, and Nate Thomas. Causal Scrubbing: A Method for Rigorously Testing Interpretability Hypotheses. Redwood Research, 2022. https://www.lesswrong.com/posts/JvZhhzycHu2Yd57RN/causal-scrubbing-a-method-for-rigorously-testing
• Judea Pearl. Causality: Models, Reasoning, and Inference (2nd ed.). Cambridge University Press, 2009.

Debugging Checklist

• Predicted divergence pre-written (fields, values, fixed companions)?
• Companion inventory complete (frozen vs. waived with arguments)?
• Null forks validated ×3 before any causal reading?
• Isolated intervention run ×3 with contracted forward traces?
• Companion-swap checks run for each live companion?
• Predicted-vs-observed scored per trial on deciding fields symmetrically?
• Verdict scoped (bundle, pins, fork point, companions) with no generality language?
• No narration, confidence, scores, agreement, single arms, or symptoms cited as cause?

What This Chapter Established

• Causal replay as predicted-vs-observed testing under companion control: three arms, swap checks, and scope-named verdicts — demonstrated on the constructed step-12 isolation, no measured runs claimed.
• The sensitivity-vs-causation separation (Chapter 40 proposes candidates; this chapter elects them) with the component-freeze variant.
• Lab 41 as a proposed causal scorecard the reader executes; the Component Freeze/Re-run Tool contract (accepts/performs/can-establish/cannot-establish/next-action).
• What was NOT proved: any cross-task causal law, any component-generality claim, or any reliability certification. One bundle tested; nothing universal.
• Research grounding: causal replay is Causal Testing (Johnson, Brun & Meliou) applied to trajectories; the metric and companion set must be fixed in writing before the run, because the interior-model version of this experiment — activation patching, the causal-mediation-analysis family (Vig et al.; ROME — Meng et al.; DAS — Geiger et al.) — flips its verdict on those choices (Zhang & Nanda) and its “isolated” effects still carry interaction terms with other components (Vaidyanathan et al. — the interior twin of companion confounding); and like causal scrubbing (Chan et al.), the method rejects a hypothesis confidently and supports one only cautiously — “H1 supported for this bundle” is the weakest verdict the evidence licenses, by design.

Next

Causes are now testable one fork at a time — but real debugging compares whole runs: the failing trajectory against the passing one, step by step, asking where they first part ways. Single-fork experiments cannot see that shape. Chapter 42, “Trajectory Diff,” aligns trajectories for differencing and names the first-divergent step as this Part’s analog of Chapter 2’s first divergence; what alignment can and cannot survive nondeterminism is its chapter’s to establish, not this one’s.