Part VIII — Building the AI Debugger

The assistant that broke every rule politely

Chapter 45 froze the past; this chapter polices the consumer. A practitioner runs a well-bundled failure through a debugging assistant that returns a report violating, in one page, every rule Chapters 01–03 established: its “observations” are paraphrases, its single hypothesis arrives with no competitor, its conclusion cites the assistant’s own confidence and one passing run, and the fix ships to production without human review. The bundle was perfect. The discipline was absent — because nothing checked it.

OBSERVATION: the report contains zero verbatim quotes resolving to bundle hashes, one hypothesis with no alternative, a confidence token (“high confidence”), a single post-fix run, and a production edit executed by the assistant. HYPOTHESIS H1 (capable-but-unchecked): the assistant can diagnose but was never constrained. H2 (incapable): no constraint set would make this report trustworthy. H3 (process-only defect): the practitioner, not the assistant, owns the missing gates. INFERENCE: none yet — H1/H2/H3 separate only when the same assistant is re-run under machine-enforced rules and its outputs are checked mechanically, not by reading tone.

This chapter’s question: which evidence-discipline rules can be encoded as machine-checkable invariants — so the debugging assistant’s output is rejected by a checker, not by a tired human’s vigilance?

Why “instruct the assistant to be careful” fails first

The obvious move — a system prompt saying “be rigorous, consider alternatives, show evidence” — fails because prose instructions are behavior requests, not enforcement. Five violations slide through politeness:

1. Paraphrase-as-observation. “The retriever returned irrelevant docs” with no verbatim quote or hash. Instruction-following produces plausible summaries; only a checker demanding hash-resolving quotes stops this.
2. Single-hypothesis reporting. One story, no competitor. “Consider alternatives” yields a token alternative (“or possibly something else”) with no distinct prediction — undiscriminating by construction.
3. Explanation-as-trace. “I determined the fault by weighing the reranker scores…” The assistant’s account of its own process cited as evidence. Process narratives are never traces.
4. Score-and-confidence citation. Reranker scores, model confidence, inter-assistant agreement, or attention patterns offered as diagnostic support. Scores rank; they do not diagnose.
5. Unverified disposal. Diagnosis flowing straight to action. A prompt cannot hold the production boundary; a separate checker with veto power can.

OPINION: a system prompt without a checker is a speed-limit sign with no radar — informative, decorative, and routinely ignored at velocity.

The mental model: invariants are the assistant’s type system. Just as a type checker rejects programs that “look right” but violate structure, the invariant checker rejects reports that read well but violate evidence structure — before any human spends judgment on them. Chapters 01–03 gave three load-bearing rules; this chapter compiles them into six machine checks.

This is literal, not metaphorical. Singhvi and colleagues’ LM Assertions are exactly this construct: a hard @Assert or soft @Suggest constraint expressed as a Python function that takes the model’s output and returns a boolean, enforced by injecting the failing output and its error message back into the prompt and retrying — which improved output-constraint conformance by up to 164% in their experiments (Singhvi et al., 2024). The six invariants below are LM Assertions on a debugging assistant. And the most basic one — that every claim comes in a labeled O/H/I record at all — is enforced not by asking but by guided decoding: Willard and Louf’s method reformulates generation as transitions through a finite-state machine so that structurally invalid output is simply unreachable (Willard & Louf, 2023).

The method: six invariants, mechanically checked

Each invariant is stated as a reject-rule over the assistant’s structured output (O/H/I records plus the bundle manifest):

1. I-1 Separation. Every claim carries exactly one label: OBSERVATION (verbatim + hash), HYPOTHESIS (with prediction), or INFERENCE (with deciding run). Check: unlabeled or doubly-labeled claims → REJECT.
2. I-2 Grounding. Every OBSERVATION resolves to a bundle hash. Check: unresolving citations → REJECT the claim, downgrade the report.
3. I-3 Plurality. ≥2 competing hypotheses, each with a predicted outcome distinct from the others. Check: one hypothesis, or two predicting the same observable → REJECT as undiscriminating.
4. I-4 Prediction-first. Each hypothesis’s prediction timestamped before its deciding run. Check: prediction missing or post-dated → INFERENCE inadmissible.
5. I-5 Trial sufficiency. Nondeterministic steps report ≥3 trials; single-run claims flagged INSUFFICIENT. Check: run-count < 3 on varying steps → verdict capped at provisional.
6. I-6 No-forbidden-evidence. No claim rests on model explanation, attention/confidence/score, correlation alone, single runs, inter-assistant agreement, or downstream symptom relief. Check: pattern-match the justification field; any forbidden source → REJECT the inference.

    flowchart TD
    R["assistant report as structured O / H / I records + the bundle manifest"] --> I1{"I-1 separation: every claim singly labeled O / H / I?"}
    I1 -->|no| REJ["REJECT — return to the assistant naming the failed invariant"]
    I1 -->|yes| I2{"I-2 grounding: every OBSERVATION resolves to a bundle hash?"}
    I2 -->|no| REJ
    I2 -->|yes| I3{"I-3 plurality: >=2 hypotheses with distinct predicted observables?"}
    I3 -->|no| REJ
    I3 -->|yes| I4{"I-4 prediction-first: each prediction timestamped before its deciding run?"}
    I4 -->|no| REJ
    I4 -->|yes| I6{"I-6: no claim rests on explanation / score / confidence / agreement / single run / symptom?"}
    I6 -->|no| REJ
    I6 -->|yes| I5{"I-5 trial sufficiency: >=3 trials on every nondeterministic step?"}
    I5 -->|no| PROV["PROVISIONAL — cap the verdict, run the missing trials"]
    I5 -->|yes| PASS["PASS — route to experiment execution + human sign-off by impact"]
  

# invariant check sketch (contract-only; illustrative, not a shipped tool)
report = load_assistant_report()  # OBSERVATION: structured O/H/I, not prose
bundle = load_manifest()          # OBSERVATION: hashes from Ch45
assert all(c.label in ("O","H","I") for c in report.claims), "I-1"
assert all(b.resolve(c.hash) for c in report.observations), "I-2"
assert len(report.hypotheses) >= 2 and distinct_predictions(report), "I-3/I-4"
assert trial_counts_ok(report, minimum=3), "I-5"
assert no_forbidden_sources(report), "I-6"  # explanation/scores/agreement/symptom
# Verdict: PASS (routable) / REJECT (returned) / PROVISIONAL (capped)

OBSERVATION (constructed illustration, not a measured run): the opening report fails I-1 (unlabeled), I-2 (no resolving quotes), I-3 (single hypothesis), I-4 (no priors), I-5 (one run), and I-6 (confidence cited) — six-for-six rejection. UPDATED BELIEF: H1 (capable-but-unchecked) stays live pending a constrained re-run; H2 unsupported until the assistant fails under enforcement too; H3 supported in part — the practitioner owned the missing checker.

No instruction-following fluency, no vendor claim about an assistant’s “reasoning capability” (attribute and date any such claim; changeable fact), and no single passing evaluation upgrades an invariant-failed report. Checkers, not adjectives.

Example: the same assistant, checked, on the second pass

Constrained to emit O/H/I records against the frozen bundle, the assistant returns: three verbatim observations (hashes resolving), H1 “stale chunk” predicting “removing chunk 4 changes the quote,” H2 “prompt ambiguity” predicting “rewording changes the quote while chunk removal does not,” predictions timestamped, three-trial plan stated, no confidence language. The checker passes I-1–I-4 and I-6, marks I-5 pending execution. The human runs the trials: chunk removal changes the quote ×3, rewording does not ×3. The machine enforced the shape; the runs supplied the content. Neither sufficed alone.

Research lineage: assertions, contracts, and where the checker lives

Hard vs. soft maps onto REJECT vs. PROVISIONAL. Singhvi and colleagues distinguish @Assert (must hold; backtrack until it does or fail) from @Suggest (should hold; note the violation and proceed) (Singhvi et al., 2024). I-1 through I-4 and I-6 are @Assert — a violation returns the report. I-5 (trial sufficiency) is closer to @Suggest — the report is not rejected, its verdict is capped at PROVISIONAL until the trials run. The backtracking mechanism — inject the failing claim and the invariant name, retry — is exactly “return to the assistant naming the failed invariant.”

The invariants are Design by Contract, aimed at the assistant. Chapter 8’s precondition / postcondition / invariant vocabulary applies directly: I-2 (grounding) is a precondition on every OBSERVATION; I-3 (plurality) is an invariant on the report as a whole; I-4 (prediction-first) is a temporal precondition on every INFERENCE. Meyer’s blame rule carries too — a report that fails I-2 is the assistant’s contract violation, not the human reviewer’s. Note the scope: the six here are report-shape invariants, the same for every failure because they come from Chapters 01–03. The system being debugged has its own domain invariants — Chapter 45’s “the quote must cite a policy section present in the snapshot,” or “refund_id present on every row entering summarize” — and Chapter 8 showed how to get them: Daikon proposes the properties that held across the passing runs as likely invariants, curated for noise, and the incident tells you which candidate was violated. Both kinds are checked the same mechanical way.

Independence is the load-bearing constraint. The checker cannot be the assistant grading itself — Chapter 44’s self-preference result (Panickssery et al.) makes an assistant a lenient judge of its own output. The checker is a separate process, ideally deterministic code rather than a second model call, and where a model is used it must not be told which system produced the report.

Lab 46: run the checker over two assistant reports (proposed)

PROPOSED, not executed: no author-measured results are reported. The evidence this chapter requires is the reader’s own check log.

Setup. Take one bundled failure and one assistant. The constraint regime (free prose vs. invariant-constrained O/H/I output) is the independent variable; failure, bundle, and assistant version (pinned) are controlled.

Task.

1. Before running, write H1/H2/H3 with distinct predicted check outcomes: H1: “constrained output passes I-1–I-4/I-6, free output fails ≥3 invariants”; H2: “both pass (invariants add nothing)”; H3: “both fail (assistant cannot emit checkable structure).”
2. Collect one report per regime; run the six checks mechanically (paper form below suffices).
3. Record PASS/REJECT/PROVISIONAL per invariant with the failing claim quoted.

Success criterion. Two check logs with per-invariant verdicts, failing claims quoted verbatim, and trial counts stated. A judgment of “the constrained one felt better” without per-invariant verdicts is explicitly not completion.

Companion tool: Diagnostic Invariant Checker

What it accepts: the assistant’s structured O/H/I report, the bundle manifest with hashes, and the trial log for any executed runs. What it performs: it applies I-1–I-6 mechanically, quotes each violating claim verbatim, marks predictions pre/post-dated, counts trials per nondeterministic step, scans justifications for forbidden sources, and emits PASS / REJECT / PROVISIONAL with the blocking invariant named. What it can establish: whether a report satisfies evidence structure, and which invariant blocks it — for the examined report only. What it cannot establish: the true cause, cross-report assistant quality, or future compliance. It never treats fluency, vendor capability claims, scores, agreement, single runs, or symptom relief as passing evidence. How its output changes your next action: PASS → route to experiment execution (Ch48) with human sign-off by impact; REJECT → return to the assistant naming the failed invariant; PROVISIONAL → execute the missing trials before any verdict.

Paper form, sufficient for this chapter:

Report ___ on bundle ___ (assistant ___ v___)
I-1 separation __ | I-2 grounding (__/__ resolve) __ | I-3 plurality __
I-4 prediction-first __ | I-5 trials (min 3: __) __ | I-6 forbidden-source scan __
VIOLATIONS (quote claim + invariant): ___  VERDICT: PASS / REJECT / PROVISIONAL

Where a software implementation does not yet exist in the reader’s stack, this record is the tool. Structure first; judgment second.

Reusable procedure: check every assistant report the same way

1. Require structure — O/H/I records against the manifest, never free prose as the decision artifact.
2. Run six checks — separation, grounding, plurality, prediction-first, trials, forbidden sources.
3. Quote violations — each REJECT names the claim and the invariant verbatim.
4. Cap verdicts — insufficient trials → PROVISIONAL maximum, regardless of plausibility.
5. Sign off by impact — high-impact actions need human verification on raw records even after PASS.

Failure modes

• Prompt-instead-of-checker. Believing instructions enforce themselves. Requests do not bind; checkers do.
• Grading prose. A human “reviewing” fluency instead of running checks. Readability is not structure.
• Checker theater. Running I-1–I-6 but overriding REJECT because “this one looks right.” Overrides without new evidence are narration with extra steps.
• Score laundering. Letting reranker scores or confidences pass as justifications. Forbidden sources stay forbidden after formatting.
• Trial-count amnesia. Passing I-5 on one run “because it was decisive.” Decisiveness is judged after repetition, not during.
• Assistant self-checking. Letting the assistant grade its own invariants. The checker is independent or it is decoration.

Limits, per contract: one check verdict covers one report on one bundle under one assistant version; it warrants structure, not truth; constraint regimes do not transfer across failures or models. UNKNOWN wherever outputs are free prose or hashes cannot be resolved.

References

• Arnav Singhvi, Manish Shetty, Shangyin Tan, Christopher Potts, Koushik Sen, Matei Zaharia, and Omar Khattab. LM Assertions: Enhancing Language Model Programs with Assertions. arXiv:2312.13382, 2024. https://arxiv.org/abs/2312.13382
• Brandon T. Willard and Rémi Louf. Efficient Guided Generation for Large Language Models. arXiv:2307.09702, 2023. https://arxiv.org/abs/2307.09702
• Bertrand Meyer. Applying “Design by Contract”. Computer 25(10), 1992, pp. 40–51. https://doi.org/10.1109/2.161279
• Arjun Panickssery, Samuel R. Bowman, and Shi Feng. LLM Evaluators Recognize and Favor Their Own Generations. NeurIPS, 2024. https://arxiv.org/abs/2404.13076

Debugging Checklist

• Report in O/H/I structure against the manifest (not free prose)?
• I-1: every claim singly labeled?
• I-2: every observation hash-resolves to the bundle?
• I-3/I-4: ≥2 competing hypotheses with distinct pre-dated predictions?
• I-5: ≥3 trials on nondeterministic steps (else PROVISIONAL)?
• I-6: no explanation/scores/agreement/single-run/symptom justifications?
• Verdict recorded as PASS / REJECT / PROVISIONAL with blockers quoted?
• High-impact action held for human verification on raw records?

What This Chapter Established

• Diagnostic AI invariants I-1–I-6: the Chapters 01–03 rules compiled into machine-checkable reject-rules (separation, grounding, plurality, prediction-first, trial sufficiency, no-forbidden-evidence) — demonstrated on a constructed six-for-six rejection, no measured runs claimed.
• The checker-before-judgment principle with the type-system mental model and the independent-checker requirement.
• Lab 46 as a proposed two-regime check log the reader executes; the Diagnostic Invariant Checker contract (accepts/performs/can-establish/cannot-establish/next-action).
• What was NOT proved: any assistant’s capability or incapability in general, any causal verdict, or any transfer across models or failures. One enforcement layer; nothing certified.
• Research grounding: the invariants are LM Assertions on a debugging assistant (Singhvi et al. — @Assert = REJECT, @Suggest = PROVISIONAL, backtracking = return-with-error, +164% constraint conformance); the O/H/I structure is enforced by guided decoding, not instruction (Willard & Louf); the invariants are Design by Contract aimed at the assistant (Meyer), with the blame rule intact; domain invariants on the system under test are mined from passing runs (Daikon, Ch8) and checked the same way; and the checker must be independent of the assistant because a model grades its own output leniently (Panickssery et al.).

Next

The rules are enforceable — but enforcement presupposes something this chapter assumed: an explicit, enumerable hypothesis space to check plurality and discrimination against. A checker cannot judge “competing” without a defined space of competitors. Chapter 47, “From Symptom to Hypotheses,” makes hypothesis generation a disciplined, enumerable procedure — coverage over cleverness.