Part VI — Debugging Prompts, Retrieval, and Hallucinations

The 40-line prompt that fails — and nobody knows which line

Chapter 30 versioned the prompt. Now the version is long: fourteen months of appended instructions, three example blocks, two guardrail paragraphs, a formatting schema, and a “be concise but thorough” contradiction nobody remembers adding. It fails on refund reference RB-8814 — invents a number — and every proposed fix appends line 41.

Concrete failure. The engineer suspects the few-shot examples. A colleague suspects the guardrail wording. A third suspects length itself. Each theory ships as an addition, the prompt grows, and the failure persists through all of them because no line has ever been removed as a test.

OBSERVATION: full prompt hash task@41ab (40 lines) reproduces the invented-reference failure on the pinned fixture across repeated trials; which subset of lines is necessary for the failure is UNKNOWN. HYPOTHESIS H1 (load-bearing example): one example block teaches the invention pattern. H2 (contradictory constraint): two instructions jointly force invention. H3 (dead weight): most lines are irrelevant and the trigger is a small core. INFERENCE: none yet — H1/H2/H3 predict different removal outcomes and are separable only by deleting content, not adding it.

This chapter’s question: what is the smallest prompt subset that still reproduces the failure — and what does it exonerate?

Why “rewrite it cleanly” fails first

The obvious move — rewriting the prompt from scratch — fails because rewrites change dozens of variables at once and teach nothing about the failing version in production. Four defects hide behind rewrite-and-hope:

1. Confounded repair. The rewrite fixes the symptom and nobody learns which line caused it. The next author re-adds the poison line because it “reads well.”
2. Regression blindness. The rewrite passes one manual check and ships. The old fixture suite (Chapter 30) was never run, so two previously green fixtures silently redden.
3. Contradiction preservation. Both halves of a contradictory pair (“always cite” + “never hedge, be concise”) survive the rewrite in softer words. The joint cause persists.
4. Single-run verdicts. The rewrite “works once” and is declared fixed. Nondeterministic generation needs repeated trials per removal step, not anecdotes.

OPINION: a prompt you cannot shrink is a prompt you do not understand. Minimization is comprehension with a paper trail.

The mental model: prompt delta-debugging — remove-until-break-applied-in-reverse (remove-until-fix, then restore-until-break). Halve the prompt, run the fixture, keep the half that still fails. The endpoint is the minimal failing prompt: every remaining line is load-bearing for the reproduction, every removed line is exonerated for this fixture.

This is Zeller and Hildebrandt’s ddmin (Chapter 1) applied to prompt text, and the mandatory pair-removal step is not an extra — ddmin handles exactly this interference case, where two elements each look innocent alone and only their combination fails (Zeller & Hildebrandt, 2002).

The method: minimize the failing prompt

Fix everything except prompt content (environment pin, index snapshot, seed/parameters, fixture input and frozen context), then:

1. Pin the repro. One failing fixture, one prompt hash, failure defined as a checkable property (e.g., “output contains a reference number absent from context”). Confirm it fails ≥3/3 before minimizing.
2. Remove in chunks. Delete halves, then quarters, then single blocks (examples, guardrails, schema, persona). After each removal, run the fixture ≥3 times.
3. Keep the failing branch. If the reduced prompt still fails, the removed content is exonerated (for this fixture); continue from the smaller file. If the failure disappears, restore and split the removed chunk.
4. Test singles and pairs. Once small, remove each remaining line alone, then each pair — pair-removal catches contradictory constraints neither line triggers solo.
5. Freeze the minimal pair. Ship two artifacts: the minimal failing prompt (diagnosis) and the minimal passing repair (smallest edit that greens the suite). Both hashed.

    flowchart TD
    P["pin one failing fixture: checkable property, confirm FAIL 3/3, freeze environment + context"] --> R["remove a chunk (halves -> quarters -> blocks); run the fixture x3"]
    R --> Q{"reduced prompt still fails 3/3?"}
    Q -->|yes| K["removed content EXONERATED for this fixture — continue from the smaller file"]
    Q -->|"now passes"| S["restore, split the removed chunk, recurse"]
    S --> R
    K --> DONE{"down to a small core?"}
    DONE -->|no| R
    DONE -->|yes| SP["remove each remaining line alone, then each pair"]
    SP --> W{"what greens the fixture?"}
    W -->|"a single line"| H1["single line convicted — targeted reword"]
    W -->|"only a pair, neither alone"| H2["contradictory pair convicted — rewrite the joint constraint"]
    W -->|"nothing; core is tiny"| H3["the rest was dead weight (H3)"]
    H1 --> G["ship the minimal failing prompt + minimal repair, both re-gated on the FULL suite"]
    H2 --> G
    H3 --> G
  

MINIMIZATION LOG (fixture: refund-RB-8814, env pinned, x3 trials per row):
task@41ab (40 lines): FAIL FAIL FAIL -> repro confirmed
- examples block (lines 12-24): FAIL FAIL FAIL -> examples EXONERATED here
- guardrail para 2 (lines 30-35): PASS PASS FAIL -> suspect (2/3 still fails? rerun x3)
rerun guardrail-removed x3: PASS PASS PASS -> failure needs guardrail para 2
- line 33 alone removed: PASS x3 | - line 31 alone removed: FAIL x3
pair (31+33) removed: PASS x3; restore 31 only: FAIL x3
MINIMAL: lines 31+33 jointly necessary (constructed illustration).
RULE: removal decides relevance. Reading decides nothing.

OBSERVATION (constructed illustration, not a measured run): the 40-line prompt fails 3/3; removing the 13-line example block still fails 3/3; removing guardrail line 33 alone passes 3/3 while line 31 alone still fails. UPDATED BELIEF: H1 exonerated for this fixture (examples removed, failure persists); H2 supported (lines 31+33 jointly necessary); H3 partly supported (13 lines dead weight). No claim beyond this fixture.

No brevity preference, fluency judgment, confidence reading, agreement across paraphrases, or downstream symptom (“support tickets dropped”) substitutes for removal outcomes.

Example: two guardrail lines that jointly invent references

The constructed refund prompt carries line 31 (“Always provide a reference number for every refund answer”) and line 33 (“Keep answers short; do not quote policy text or say information is missing”). Alone, line 31 with full context still grounds (the model quotes the pending-refund line). Alone, line 33 hedges (“no reference issued yet”). Together, they forbid both grounding and hedging — invention is the only compliant completion:

# prompt minimization harness sketch: chunk, run, classify (env fixed)
prompt = load("prompts/system@v15.md")  # OBSERVATION: 40 lines, hash task@41ab
fixture = load_fixture("refund-RB-8814")  # frozen context WITHOUT RB-8814
# H1 probe: remove examples -> predict FAIL persists. H2 probe: remove line 33
# -> predict PASS. H3 probe: remove persona/schema -> predict FAIL persists.
for candidate in halve_then_quarter(prompt):
    results = [run(candidate, fixture, seed=s) for s in (1, 2, 3)]
    verdict = "FAIL-persists" if all(fails(r) for r in results) else "unfixed-or-flaky"
    log(candidate.hash, removed_lines(candidate), verdict)  # MEASUREMENT per trial
# Pair pass over survivors: remove each pair, x3 trials, pre-written predictions.

In the constructed case the harness exonerates 30 of 40 lines, convicts the 31+33 pair, and the repair is minimal: replace line 31 with “Provide a reference number only if one appears verbatim in context; otherwise state none was issued.” The repaired prompt is re-run against the full Chapter 30 suite (not just the one fixture) before shipping — a minimal fix that breaks two other fixtures is a trade, not a repair.

Second artifact: the exoneration ledger

Minimization produces two outputs and teams routinely keep only one (the fix). The exoneration ledger — lines removed without effect, with trial data attached — is the output that compounds, because it answers next quarter’s “can we add a persona paragraph?” with evidence instead of debate:

1. Log every exonerated block with its verdict line. lines 12–24 (examples): removed, FAIL persists 3/3 → EXONERATED for fixture refund-RB-8814, env pin idx-2026-08-14. A removal without trial counts is marked CLAIM, not finding.
2. Scope every entry. Exoneration holds for the fixture and pin listed, nothing else. The ledger’s header states this in one sentence so future readers cannot inflate it.
3. Re-check on pin change. New model identifier or index snapshot re-opens the ledger for the failing fixtures; dead weight under one pin can be load-bearing under another (phrasing sensitivities move with models — OBSERVATION from re-runs, not theory).
4. Mine the ledger for style rules. When three minimizations exonerate persona paragraphs and convict guardrail pairs, the team earns an evidence-backed style constraint (“guardrails state the forbidden completion explicitly”) — proposed from data, still gated on the suite.

EXONERATION LEDGER (constructed illustration, fixture refund-RB-8814):
lines 01-04 (persona): removed FAILx3 -> EXONERATED (this fixture/pin)
lines 12-24 (examples): removed FAILx3 -> EXONERATED (this fixture/pin)
lines 36-40 (schema): removed FAILx3 -> EXONERATED (this fixture/pin)
lines 31+33 (guardrail pair): removed PASSx3 -> CONVICTED JOINTLY
RULE: exonerated-here is not harmless-everywhere. Scope lines are load-bearing text.

OBSERVATION (constructed illustration): the ledger from two minimizations lets a later author delete 11 lines on sight with suite-green confirmation, cutting a debug session from a day to an hour. UPDATED BELIEF: ledger supported as accelerator for this team-instance; no claim that exoneration transfers across fixtures. Time saved is process evidence, not diagnostic proof.

Research lineage: most of the prompt is dead weight, and contradictions are invisible

Prompts compress dramatically, which means H3 is usually partly right. Jiang and colleagues’ LLMLingua uses a small model to score each token’s contribution and drops the low-contribution ones, reaching up to 20x compression with little performance loss on reasoning and summarization tasks (Jiang et al., 2023). That is strong evidence that a long production prompt carries a lot of inert text. Two caveats keep it from replacing this chapter’s method: LLMLingua removes by information content, not by causal necessity for a specific failure, and it is tuned for cost, not diagnosis. The minimization here keeps the lines that are load-bearing for the reproduction — a different and smaller set.

A contradictory instruction pair has no reliable resolution, so the model can produce neither option. Wallace and colleagues formalized an instruction hierarchy (system over user over tool output) and showed that without explicit training models treat conflicting instructions as roughly equal in authority (Wallace et al., 2024). Even the widely-used system/user prompt separation does not reliably establish a hierarchy: Geng and colleagues found priority-adherence rates on simple formatting conflicts varying widely by model and often well below half, with models carrying strong inherent biases toward particular constraint types regardless of which was designated higher priority (Geng et al., 2026). The “priority” the engineer thinks they set is frequently not the one the model acts on. When line 31 says “always give a reference” and line 33 forbids the hedge that would let the model comply honestly, there is no principled winner — and invention becomes the only completion that satisfies both surface constraints. The model will not report this conflict; pair-removal is how you find it.

Lab 31: remove-until-break with pre-written removal predictions (proposed)

PROPOSED, not executed: no author-measured results are reported. The evidence this chapter requires is the reader’s own minimization log.

Setup. Take one versioned prompt (Chapter 30 record) with one fixture that fails ≥3/3 under a pinned environment. Freeze the environment, the fixture input, and the frozen context. The prompt subset is the independent variable; everything else is controlled.

Task.

1. Before removing anything, write H1/H2/H3 with distinct predicted removal patterns: H1: “removing lines – greens the fixture (example-driven)”; H2: “no single-line removal greens it, but pair (+) does (contradiction)”; H3: “failure persists down to ≤___ lines (small core, rest dead weight).”
2. Minimize top-down (halves → quarters → singles → pairs), ≥3 trials per candidate. Record OBSERVATION (per-trial fail/pass verbatim) and UPDATED BELIEF after each step.
3. Confirm the endpoint: the minimal failing prompt fails 3/3, and removing any one remaining line (or the convicted pair) passes 3/3.

Success criterion. A minimization log ending in a minimal failing prompt plus a minimal passing repair, both hashed, both run against the full suite ×3. A rewritten prompt with no removal log is explicitly not completion.

Companion tool: Prompt Minimization Harness

What it accepts: a versioned prompt, one pinned failing fixture with frozen context and checkable failure property, the environment pin, and the removal plan (chunk order, trial count). What it performs: it generates prompt subsets in delta-debugging order, runs each subset the declared number of trials with everything else fixed, classifies each candidate (fail-persists / greens / flaky-split), and enforces single- and pair-removal confirmation before declaring minimality. What it can establish: which lines (or line pairs) are necessary for the failure on the examined fixture — and which are exonerated — under the pinned environment. What it cannot establish: necessity across fixtures, environments, or models; or that the minimal repair is globally safe. It never treats readability, confidence, agreement, single-run outcomes, scores, or downstream symptoms as removal evidence. How its output changes your next action: a convicted single line routes to targeted rewording; a convicted pair routes to contradiction repair (rewrite the joint constraint); full exoneration of the suspected block routes to pipeline probes (Chapters 32–33); a minimal repair that reddens other fixtures routes back to the Chapter 30 suite gate.

Paper form, sufficient for this chapter:

Fixture: ___  Env pin: ___  Start hash: ___ (___ lines)
Removals (hash | lines cut | x3 verdict): ___ / ___ / ___
MINIMAL FAILING: hash ___ (lines ___)  MINIMAL REPAIR: hash ___
Full suite on repair: ___/___ x3  NEXT: ship / revise / escalate to pipeline

Where a software implementation does not yet exist in the reader’s stack, this record is the tool. Delete before theorizing.

Reusable procedure: minimize every failing prompt

1. Pin one failing fixture — checkable property, 3/3 repro, environment frozen.
2. Remove top-down — halves, quarters, singles, then pairs; ≥3 trials per candidate.
3. Keep fail-persists branches — exonerate what removal does not need.
4. Confirm minimality — every survivor line (or convicted pair) individually necessary.
5. Re-gate the repair — minimal fix against the full suite, never the single fixture alone.

Failure modes

• Additive fixing. Appending instructions instead of removing suspects. Growth hides the poison.
• Rewrite confounding. Changing everything at once. Attribution destroyed.
• Single-trial removal. One pass after a cut “proving” irrelevance. Flakiness mimics exoneration; three trials minimum.
• Single-line blindness. Testing lines only solo and missing contradictory pairs. Pairs are mandatory.
• Fixture narcissism. Minimizing to one fixture while the suite reddens elsewhere. The suite is the gate, not the fixture.
• Unfrozen context. Minimizing while retrieval results drift between runs. Freeze the context or the log is noise.
• Ledger-less minimization. Keeping the fix and discarding the exoneration record. The next author re-suspects the exonerated block at full cost.
• Pin drift mid-minimization. A model or snapshot change halfway through the removal series. Early removals and late removals stop being comparable; restart the series.

Limits, per contract: one minimization covers one prompt version, one fixture, one environment pin; it does not generalize to other fixtures or models, and stays UNKNOWN where trials were single or the context unfrozen.

References

• Andreas Zeller and Ralf Hildebrandt. Simplifying and Isolating Failure-Inducing Input. IEEE Transactions on Software Engineering 28(2), 2002, pp. 183–200. https://doi.org/10.1109/32.988498
• Huiqiang Jiang, Qianhui Wu, Chin-Yew Lin, Yuqing Yang, and Lili Qiu. LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Models. Proceedings of EMNLP, 2023, pp. 13358–13376. https://doi.org/10.18653/v1/2023.emnlp-main.825
• Eric Wallace, Kai Xiao, Reimar Leike, Lilian Weng, Johannes Heidecke, and Alex Beutel. The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions. arXiv:2404.13208, 2024. https://arxiv.org/abs/2404.13208
• Yilin Geng, Haonan Li, Honglin Mu, Xudong Han, Timothy Baldwin, Omri Abend, Eduard Hovy, and Lea Frermann. Control Illusion: The Failure of Instruction Hierarchies in Large Language Models. Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), 2026 (arXiv:2502.15851). https://arxiv.org/abs/2502.15851

Debugging Checklist

• One failing fixture pinned (checkable property, 3/3 repro, env frozen)?
• H1/H2/H3 removal predictions pre-written with distinct patterns?
• Removals run top-down (chunks → singles → pairs), ≥3 trials each?
• Every removed block classified (exonerated vs. suspect) from trial data?
• Minimal failing prompt confirmed (each survivor necessary, pairs tested)?
• Minimal repair re-run against the full suite (not the single fixture)?
• Exoneration ledger kept (removed blocks with trial verdicts)?
• Environment pin unchanged through the whole removal series?
• No readability, confidence, agreement, scores, single runs, or symptoms cited as verdict?

What This Chapter Established

• Prompt delta-debugging: remove-until-fix then restore-until-break, with mandatory pair-removal, demonstrated on the constructed 40-line refund prompt (31+33 jointly convicted), no measured runs claimed.
• The exoneration record as a first-class output: removed-without-effect lines are findings, not waste.
• Lab 31 as a proposed minimization log the reader executes; the Prompt Minimization Harness contract (accepts/performs/can-establish/cannot-establish/next-action).
• The exoneration ledger as the compounding output: scoped removal verdicts that accelerate the next minimization.
• What was NOT proved: any prompt-writing rule, any cross-fixture generality, or any model-capability claim. One prompt minimized for one fixture; nothing universal.
• Research grounding: the method is ddmin on prompt text, and pair-removal is ddmin’s interference case (Zeller & Hildebrandt); prompts carry heavy dead weight — compressible up to 20x with little loss (LLMLingua), though by information content, not causal necessity; and a stated instruction priority (system over user) is not reliably enforced (Wallace et al.; Geng et al. — adherence often well below half, with inherent constraint-type biases), so a contradictory pair has no principled winner, the model produces neither compliant option, and it never reports the conflict — pair-removal is the only way to see it.
• Position in the arc: Chapter 30 versioned the program; this chapter shrank the failing case to its load-bearing core. Core frozen, suspects named.

Next

A minimal prompt still fails through no fault of its words when the evidence it needs never arrives — the retrieved context is itself assembled by a multi-stage machine with its own failure points. Chapter 32, “Retrieval Is a Pipeline,” maps chunk, embed, index, and rank stages with per-stage contracts; which stage broke first is its chapter’s to establish, not this one’s.