Part VII — Debugging Agents

Single generations are over — the debugging object is now the trajectory

Part VI treated the prompt as a program and pinned every claim to either retrieval or generation. That discipline holds for one call. It breaks the moment the system makes five of them: a support agent looks up order 8841, checks the refund policy, calls issue_refund, re-checks the balance, and apologizes — twice refunding the customer while its final message reads perfectly. The generation is clean. The trajectory is the defect.

OBSERVATION: tool-call log shows issue_refund(order=8841, amount=42.00) at step 4 with {status: ok} and again at step 7 with {status: ok}; the final message mentions one refund. Ledger shows two charges. HYPOTHESIS H1 (lost observation): step 4’s confirmation never entered the agent’s working state, so step 7 re-issued. H2 (bad plan): the plan always contained two refunds (retry-as-procedure). H3 (state overwrite): an intermediate summarization step dropped the step-4 result before step 7 decided. INFERENCE: none yet — H1/H2/H3 predict different trajectory records and are separable only by reading steps, not the final text.

This chapter’s question: what is the trajectory, exactly — and what must you record at each step before any diagnosis is licensed?

Why “read the answer” fails first

The obvious move — judging the final message — fails because multi-step behavior fails between generations, not inside one. Four defects hide behind output-only debugging:

1. Inter-step loss. Each generation can be locally coherent while the handoff between steps drops the one field that mattered (the refund confirmation ID).
2. Narrator trust. The agent’s closing summary (“refunded once”) is behavior — generated text optimizing plausibility — not a read of its own tool log. Self-narration is never trajectory evidence.
3. Silent re-execution. Retried tool calls with identical arguments look like diligence in prose and like double-spend in the ledger. Only argument-level records separate them.
4. Single-run storytelling. One clean re-run “proves” the double-refund was flakiness. Nondeterministic agents need repeated trajectory repros, not anecdotes.

OPINION: debugging an agent from its final answer is like debugging a pipeline from its exit banner — you are reading the press release, not the log.

The mental model: an agent is a trajectory — an ordered sequence of steps, each binding state, action, and observation. The debugging object changes from Evidence (Part VI’s retrieved-vs-generated claim) to Trajectories (state → action → observation → new state, repeated). First divergence still rules, but it now points at a step index, not a token span.

This is the ReAct loop — reason, act, observe — made auditable: Yao and colleagues’ whole argument for interleaving actions with observations was that grounding each decision in a real observation reduces the error propagation that pure reasoning chains suffer (Yao et al., 2023). The five-field schema below is what it takes to check that grounding actually happened. And the schema maps onto a formal decomposition: Sumers and colleagues’ CoALA framework separates an agent’s working memory (current context), its action space (external grounding actions vs. internal retrieval / reasoning / learning), and its decision cycle (Sumers et al., 2024). In that vocabulary, H1 (lost observation) is a failure to write an observation into working memory, and H3 (state overwrite) is an internal learning action corrupting it.

The method: define the step, then read the sequence

A trajectory is a list of step records. Each step MUST contain five fields; a missing field suspends diagnosis for that span:

1. Step index and kind. think | plan | act (tool call) | observe | repair | answer. Kinds are labels for triage, not explanations of intent.
2. Action with verbatim arguments. Tool name plus exact args (issue_refund(order=8841, amount=42.00)), not paraphrases.
3. Observation verbatim. The tool’s actual return ({status: ok, refund_id: r-991}), including errors, truncation flags, and latency. Paraphrased observations are narration, not evidence.
4. State delta. What changed in working state after the observation: added/removed/overwritten keys, with before/after hashes where feasible. No delta recorded → state claims are UNKNOWN.
5. Branch source. Which prior steps the current decision consumed (explicit context references), so a dropped confirmation is visible as an unconsumed observation.

This step record is the trajectory slot of the book’s one diagnostic-case record — the same object that was a pinned bundle in Part IV and a case file in Part V, and that Chapter 45 freezes as the AI crash dump. Each Part adds fields; none invents a new format.

    flowchart TD
    F["freeze the run: log, tool versions, initial state, external success check (ledger / hash / count)"] --> SC{"every step carries the 5 fields: kind, verbatim args, verbatim observation, state delta, branch source?"}
    SC -->|no| U["schemaless span — instrument first; diagnosis suspended here"]
    SC -->|yes| E["draw consumed-by edges: link each observation to the steps that used it"]
    E --> FD["name the first-divergent step: earliest index departing from a progressing trajectory"]
    FD --> W{"records shape at that step?"}
    W -->|"observation never consumed, action repeated later"| H1["H1 lost observation — Ch37 observation plumbing"]
    W -->|"plan listed the repeated action before any observation"| H2["H2 bad plan — Ch38 taxonomy"]
    W -->|"a compress / summarize step dropped a key present in its input"| H3["H3 state overwrite — Ch37 compression"]
    H1 --> RR["re-run the frozen prefix x3 before any causal claim"]
    H2 --> RR
    H3 --> RR
  

STEP SCHEMA (minimal; every step, no exceptions):
step 4 | act: issue_refund(order=8841, amount=42.00)
  -> observe: {status: ok, refund_id: r-991} (latency 812ms, truncated: no)
  -> state-delta: +refunds_issued[r-991]; balance 120.00 -> 78.00
  -> consumed-by: (to be filled by later steps; empty here = SUSPECT)
step 7 | act: issue_refund(order=8841, amount=42.00)
  -> observe: {status: ok, refund_id: r-992} (latency 790ms)
  -> state-delta: +refunds_issued[r-992]; balance 78.00 -> 36.00
RULE: an unconsumed observation at step N followed by a repeated
action at step N+k is H1-shaped (lost observation) until the plan
record proves H2 or the summary record proves H3.

OBSERVATION (constructed illustration, not a measured run): step 4’s refund_id: r-991 never appears in any later step’s consumed inputs; step 7 repeats the identical call; the final answer cites one refund. UPDATED BELIEF: H1 supported for this instance; H2 live-or-exonerated pending the plan record (does any plan step list two refunds?); H3 live-or-exonerated pending the summary record (did a compress step drop r-991?).

Example: the double refund, read as trajectory

The ledger complaint arrives. The practitioner does not re-prompt (“be careful with refunds”). She freezes the log and walks the schema:

# trajectory reading: freeze, schema-check, then hypothesize (no repair yet)
steps = load_trajectory(run_id)  # step records with args/returns/deltas
for s in steps:
    assert_has(s, ["index", "kind", "args_verbatim", "observation_verbatim",
                   "state_delta", "latency"])  # OBSERVATION: which steps are schemaless?
# H1 probe: observations never consumed downstream -> lost-observation spans
# H2 probe: plan steps listing repeated actions before any observation -> bad-plan span
# H3 probe: summarize/compress steps whose output drops keys present in input -> overwrite span
# First-divergent step: earliest index where observed state/action diverges
# from a progressing trajectory. Predictions pre-written per hypothesis.

In the constructed case the walk returns: plan (step 1) lists one refund; step 4’s observation is complete; no summary step runs between 4 and 7; step 7’s inputs contain order and amount but not r-991. First divergence: step 7, where a progressing trajectory would branch on refunds_issued non-empty. H1 stands for this instance; H2/H3 exonerated here — by records, not by the agent’s apology.

No confidence value on either generation, no agreement across re-runs, no single clean retry, and no downstream symptom (“customer stayed, so fine”) substitutes for the consumed-by column. Steps decide; prose does not.

Research lineage: state-based verification and the reliability floor

Judge the world state, not the transcript. τ-bench evaluates a tool-using agent by comparing the database state at the end of the conversation against an annotated goal state — the double-refund ledger is exactly this kind of check (Yao et al., 2025). A conversation that reads perfectly while the database is wrong is the normal case the transcript cannot show; the success criterion must be an external state check. Its successor τ²-bench extends the check to dual-control settings where the user and the agent both call tools and mutate the database — the external state check then has to account for more than one writer.

The ≥3-trial rule is calibrated for a low-reliability regime. τ-bench also introduced pass^k — the fraction of tasks an agent solves on all k independent attempts — and found that even strong function-calling models scored pass^8 below 25% on retail tasks (Yao et al., 2025). An agent that succeeds once and fails on re-run is not an anomaly; it is the median. That is why a single clean re-run is one trial, never an exoneration.

Multi-agent systems fail in more ways, and it has been catalogued. For systems where several agents coordinate, Cemri and colleagues’ MAST taxonomy identifies fourteen recurring failure modes; across more than 1,600 annotated traces from seven frameworks, failures split roughly into specification and system-design problems (~42%), inter-agent misalignment (~37%), and verification and termination failures (~21%), with per-framework failure rates of 41–87% (Cemri et al., 2025). It is a forward reference for the multi-agent chapters, and a reminder that the single-agent trajectory here is the simplest case.

First-divergent is not always first-defective. As at token scale (Chapter 2), the first-divergent step is where a progressing trajectory departs — which may be the first consequence of an earlier fault rather than the fault itself. The step index localizes; the discriminating probe convicts.

Lab 36: trajectory-vs-output with pre-written step predictions (proposed)

PROPOSED, not executed: no author-measured results are reported. The evidence this chapter requires is the reader’s own frozen trajectory.

Setup. Take one failed multi-step run with an exportable tool-call log (actions, args, returns) or instrument a fresh one on a task with a verifiable side effect (refund ledger, file hash, row count). Freeze the log, tool versions, and initial state. The evidence granularity (final-output-only vs. full step records) is the independent variable; task, model, seed, and environment are controlled.

Task.

1. Before reading steps, write H1/H2/H3 with distinct predicted step signatures: H1: “step N observation absent from all later consumed inputs, repeated action at N+k”; H2: “plan step lists the repeated action before any observation”; H3: “compress/summarize step whose output drops a key present in its input.”
2. Judge the run from the final answer alone; record the verdict (and its basis) as the output-only baseline.
3. Re-judge from full step records using the schema; locate the first-divergent step index.

Re-run the frozen prefix ≥3 times (model/seed/context fixed) to separate a trajectory defect from sampling noise; a single clean re-run is one trial, verdict UNKNOWN until the set completes.

Success criterion. A schema-checked step table with the first-divergent step named plus per-run signature results. A final-answer judgment or single-retry story is explicitly not completion.

Companion tool: Trajectory Overview Viewer

What it accepts: the frozen trajectory (ordered step records with kinds, verbatim args/returns, state deltas, latencies) plus the success criterion (e.g., “exactly one refund, ledger-verified”). What it performs: it renders every step in the five-field schema, flags schemaless steps, draws consumed-by edges from observations to later decisions, and marks the first-divergent step where observed state/action departs from a progressing trajectory. What it can establish: whether the failure is localizable to a step span, where the first divergence sits, and which hypothesis shape the records support — for the examined run only. What it cannot establish: why the agent dropped or repeated (prompt vs. context vs. model internals need later-chapter probes), generality across tasks, or future reliability. It never treats the agent’s self-narration, confidence values, agreement, single-run outcomes, or downstream symptoms as trajectory evidence. How its output changes your next action: H1 routes to observation-plumbing inspection (Chapter 37); H2 routes to plan-record inspection (Chapter 38); H3 routes to compression/summarization inspection (Chapter 37); schemaless spans route to instrumentation before any repair.

Paper form, sufficient for this chapter:

Run: ___  Criterion: ___ (ledger/hash/count ___)
Steps (n=___): schemaless ___ (indices ___) | first divergence: step ___
Consumed-by gaps: ___ (obs at ___ unconsumed)  VERDICT: H1 / H2 / H3 / UNKNOWN
NEXT: Ch37 instrumentation / Ch38 taxonomy / wider trials

Where a software implementation does not yet exist in the reader’s stack, this record is the tool. Schema before sentencing.

Reusable procedure: promote every agent failure to a trajectory

1. Freeze — run ID, log, tool versions, initial state, success criterion with its external check.
2. Schema-check — every step carries the five fields; mark schemaless spans UNKNOWN.
3. Draw consumption — link each observation to the steps that consumed it; gaps are suspects.
4. Name first divergence — earliest step departing from a progressing trajectory.
5. Hypothesize by shape — H1/H2/H3 from records, predictions pre-written, ≥3 trials before verdict.

Failure modes

• Output-only review. Judging the answer. Trajectories fail between generations; endpoints hide them.
• Narrator trust. “The agent explained it only refunded once.” Self-narration is behavior, not trajectory — the ledger and the log outrank the apology.
• Schemaless sentencing. Diagnosing spans with missing args, returns, or deltas. Missing fields mean UNKNOWN, not “probably fine.”
• Single-run exoneration. One clean retry closing a double-spend. Luck is not a fix; three trials minimum.
• Multi-fix confounding. Changing prompt, tools, and model after one failure. One intervention per trajectory series.
• Score worship. Citing confidence, judge scores, or inter-run agreement as the verdict. Signatures decide; scores narrate.

Limits, per contract: one trajectory analysis covers one run under one tool/seed/state revision; it does not explain causes, does not certify the agent, and does not transfer across tasks. UNKNOWN wherever step records are missing or paraphrased.

References

• Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. ReAct: Synergizing Reasoning and Acting in Language Models. International Conference on Learning Representations (ICLR), 2023. https://arxiv.org/abs/2210.03629
• Theodore R. Sumers, Shunyu Yao, Karthik Narasimhan, and Thomas L. Griffiths. Cognitive Architectures for Language Agents. Transactions on Machine Learning Research (TMLR), 2024. https://arxiv.org/abs/2309.02427
• Shunyu Yao, Noah Shinn, Pedram Razavi, and Karthik Narasimhan. τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains. International Conference on Learning Representations (ICLR), 2025 (arXiv:2406.12045). https://arxiv.org/abs/2406.12045
• Mert Cemri, Melissa Z. Pan, Shuyi Yang, et al. Why Do Multi-Agent LLM Systems Fail? arXiv:2503.13657, 2025. https://arxiv.org/abs/2503.13657

Debugging Checklist

• Run frozen (log, tool versions, initial state, external success check)?
• Every step schema-checked (kind, verbatim args, verbatim observation, state delta, latency)?
• Consumed-by edges drawn; unconsumed observations named?
• Success criterion is an external world-state check (ledger/db/hash), not the transcript?
• First-divergent step index named (not a token span)?
• H1/H2/H3 signatures pre-written with distinct predicted step patterns?
• Trajectory re-run ≥3 times (all else fixed) before any causal claim?
• No self-narration, confidence, agreement, single runs, or symptoms cited as verdict?

What This Chapter Established

• The trajectory as the first-class debugging object: ordered step records binding state, action, and observation — demonstrated on the constructed double-refund case, no measured runs claimed.
• The five-field step schema (kind, verbatim args, verbatim observation, state delta, branch source) with the schemaless-means-UNKNOWN rule.
• The first-divergent-step method (first divergence pointed at a step index) separating H1/H2/H3 shapes by records.
• Lab 36 as a proposed trajectory-vs-output record the reader executes; the Trajectory Overview Viewer contract (accepts/performs/can-establish/cannot-establish/next-action).
• What was NOT proved: any cause of the dropped observation, any agent capability claim, or any certification of a task family. One trajectory framed; nothing universal.
• Research grounding: the trajectory is the ReAct loop made auditable (Yao et al. 2023), and the five-field schema maps onto CoALA’s working-memory / action-space / decision-cycle decomposition (Sumers et al.) — H1/H3 are working-memory write failures; the success criterion must be an external state check — extended to dual-writer settings by τ²-bench — and the ≥3-trial rule is calibrated for a regime where even top agents score pass^8 below 25% (τ-bench, Yao et al. 2025); multi-agent failure has its own 14-mode taxonomy (MAST, Cemri et al. — ~42% specification/design, ~37% inter-agent misalignment, ~21% verification across 1,600+ traces); and the first-divergent step localizes, it does not by itself convict.

Next

The trajectory is defined but mostly unobservable in practice — steps arrive paraphrased, observations truncated without flags, state deltas missing, latencies unlogged. Naming the first divergence means little if half the steps are schemaless. Chapter 37, “Trace the Agent,” specifies the instrumentation contract that makes trajectories readable: what to log per step, and what verdicts each missing field forbids; what that contract costs in practice is its chapter’s to establish, not this one’s.