Part IV — Debugging Models

The upgrade that fixed everything (except the refunds)

Chapter 22 closed with a repaired veer on revision A. Then the upgrade lands: revision B, announced with higher scores, migrated over a weekend. Monday the refund fixture drops from 11/12 to 7/12 — different cases fail, the citation veers at a new token, support reopens tickets the team had closed. Nothing in the pipeline changed; the weights did. Nobody ran the old behaviors against the new weights before switching, because nobody had the old behaviors pinned as an asset.

Concrete failure. The pinned split-shipment suite (12 cases + boundary bundles + sampling configs from Chapters 17–21) was never executed against revision B. Post-migration forensics show 3 newly failing cases (all citing the general policy where A cited 4.2), 1 newly passing case, and 8 unchanged — a net regression wearing an upgrade’s announcement. The team debates rolling back on anecdotes because the diff was never measured.

OBSERVATION: same 12-case fixture, same bytes, same deterministic params: rev-A 11/12, rev-B 7/12 (MEASUREMENT, ≥5 trials per case, both revisions pinned). HYPOTHESIS H1 (behavioral regression): rev-B genuinely decides 3 cases worse — the weights moved the boundary; rollback or repair justified. HYPOTHESIS H2 (harness drift): the fixture, template, or params shifted under the comparison — the “regression” measures the harness, not the weights. HYPOTHESIS H3 (redistributed competence): rev-B trades 3 old cases for genuinely new capability elsewhere — the diff is real but the verdict needs the full suite, not the anecdote. INFERENCE: none yet — only a pinned-suite, both-revisions, diff-first comparison with pre-written movement FORECASTs separates weight movement from harness drift from trade.

This chapter’s question: when the model itself changes, what measured diff justifies staying, rolling back, or repairing — before production votes with tickets?

Why “the new model scores higher” fails first

The obvious move — accepting the upgrade on its announced scores — fails because headline scores measure someone else’s fixture under someone else’s harness. Four diff traps:

1. Score-as-diagnosis. A higher global number hiding a collapsed critical slice — Chapter 16’s disease returning in upgrade clothes. The refund slice is 3 cases the headline never contained.
2. Unpinned comparison. A vs. B run with different templates, params, or seeds. The diff then measures the harness gap, and every conclusion is UNKNOWN with extra steps. Biderman and colleagues, summarizing three years of running the Language Model Evaluation Harness, report that small implementation differences — prompt phrasing, answer extraction, score normalization — routinely swing benchmark numbers by margins that swamp real model differences (Biderman et al., 2024). The harness-hash guard below is their central lesson made executable.
3. Single-failure rollback. One new failure reverting an upgrade that fixed ten. Diff discipline counts both directions or it is anecdote management.
4. Silent prompt-repair. Quietly re-tuning prompts post-upgrade until the suite passes, then calling the weights “fine.” The suite now measures prompt+B jointly; the weight diff itself was never recorded and the next upgrade repeats the blindness.
5. Coverage freeze. Running the same 12 cases for a year while the product’s question distribution moves on. The diff stays green because the suite stopped describing production — coverage reviews belong on the same calendar as the upgrades.

OPINION: every model revision is a new system wearing an old name. Treat it like a dependency major-version bump: pinned suite, measured diff, rollout gates — or accept tickets as your test suite. The diff table is the cheapest insurance in Part IV.

The mental model: the behavioral diff as the upgrade’s contract. The suite (fixtures + bundles + params + seeds from Chapters 17–21) runs byte-identical against both revisions; the diff table (fixed / broken / newly-passing / unchanged per case, with trial counts) is the deliverable. “Representation” earns its title-word here in the narrow, honest sense: whatever changed inside the weights is observable to practitioners only as this behavioral movement — no interior claim is made or needed.

The method: diff-first rollout

1. Pin the suite. The regression asset: fixtures (12 refund cases + slice cases from earlier chapters), frozen bundles (bytes, template version, tokenizer, params, seeds), and pass criteria. Hashed, versioned, runnable without the author present.
2. Run both revisions byte-identical. Same suite, same harness, same seeds, ≥5 trials per case per revision. Record per-case outcomes (pass/fail counts), not just totals — the diff lives at case granularity.
3. Read the diff against FORECASTs. H1: ≥3 cases flip pass→fail with harness hashes equal — regression. H2: harness hashes differ or the flip vanishes under harness repair — drift, rerun. H3: flips in both directions with the new-pass cases clustering on a capability the suite under-covers — trade, expand the suite before verdicting.
4. Gate the rollout on the diff. Green (no pass→fail + newly-passing recorded): roll forward with the diff filed. Yellow (localized regression): hold the slice on rev-A, ship rev-B elsewhere, repair the slice. Red (broad regression): roll back, file the diff as the incident’s first artifact. Every gate decision cites case rows, never headlines.

    flowchart TD
    S["pin the suite: fixtures + bundles + params + seeds, hashed"] --> HH{"harness hash equal for both revisions?"}
    HH -->|no| H2["H2: harness drift — repair, rerun; the diff is UNKNOWN until hashes match"]
    HH -->|yes| R["run both revisions byte-identical, >=5 trials per case"]
    R --> D["per-case diff: unchanged / fixed / BROKEN / new-pass"]
    D --> G{"diff pattern?"}
    G -->|"no pass->fail; some new-pass"| GR["GREEN: roll forward, file the diff as the new baseline"]
    G -->|"localized slice regression"| YE["YELLOW: hold that slice on rev-A, ship rev-B elsewhere, repair offline"]
    G -->|"broad pass->fail"| RE["RED: roll back, file the diff as the incident's first artifact"]
    G -->|"flips both ways, new-pass cluster off-suite"| EX["H3: redistributed competence — expand the suite before any verdict"]
  

# diff-first rollout (harness frozen; only the revision moves)
suite = load_suite("refund-regression-v3")  # fixtures + bundles + seeds, hashed
assert suite.harness_hash == current_harness()  # H2 guard: harness drift voids the diff
res_A = run_suite(suite, model_rev="rev-A", trials=5)  # MEASUREMENT per case
res_B = run_suite(suite, model_rev="rev-B", trials=5)  # MEASUREMENT per case
print(diff_table(res_A, res_B))  # per case: unchanged / fixed / BROKEN / new-pass
# FORECAST: H1: >=3 pass->fail, harness equal; H2: flip vanishes on harness repair;
# H3: flips both directions + new-pass cluster. Else UNKNOWN, expand suite.

OBSERVATION (constructed illustration, not a measured run): harness hashes equal; per-case diff: 8 unchanged, 3 pass→fail (all 4.2-citation cases), 1 fail→pass; deterministic params, 5 trials each, flips stable across trials. UPDATED BELIEF: H1 supported for the refund slice — rev-B regresses the exact behavior Chapter 22 repaired; H3 noted (1 new pass) but unassessed beyond this suite — suite-expansion required before any trade claim. INFERENCE: yellow gate — hold refund path on rev-A, file the 3-row diff, repair against rev-B offline; no prompt-repair on the production path until the weight diff is recorded.

Note why yellow exists as a gate at all. Green-or-red would force a false choice: ship a known slice regression, or forfeit the genuine new pass plus every unmeasured improvement off-suite. Yellow records both, holds the blast radius to the regressed slice, and converts the 3 broken rows into repair fixtures with the diff as their provenance. The gate is a decision with an address, not a feeling about the release notes.

Research lineage: net-neutral upgrades still churn, and it has a fix

Equal accuracy hides per-example flips. Milani Fard and colleagues named this prediction churn: successive model iterations change the prediction on many individual examples without changing aggregate accuracy, which makes it hard to tell whether a release is actually an improvement — and which is exactly why the diff table is per-case, not per-total (Milani Fard et al., 2016). Their mitigation, regularizing the new model toward the old one, is a prevention worth knowing for internally-trained models.

The pass→fail flip has a name and a training-time countermeasure. Yan and colleagues call a case the new model breaks that the old model handled a negative flip, measure it as the negative-flip rate, and show that positive-congruent training — adding a loss term that penalizes new errors on old successes — reduces regression without hurting accuracy (Yan et al., 2021). The “BROKEN” column of the diff table is the negative-flip set for this suite. For language models specifically, Echterhoff and colleagues’ MUSCLE carries the same idea forward: instance regression shows up even when the fine-tuning procedure is unchanged, and a trained compatibility adapter cut negative flips by up to about forty percent on a Llama 1 → Llama 2 update (Echterhoff et al., 2024) — a number bounded to that update pair, but a mechanism worth knowing for models you train.

No interior signal tells you which cases will flip. Sheng and Lu tested the obvious shortcuts — new-model confidence, logit margin, attention entropy, and cross-version divergence between old and new outputs — as predictors of which samples a version update would break, and found none works across tasks and update pairs; the best predictor changes with the task (Sheng & Lu, 2026). There is no cheap signal that substitutes for running the full pinned suite per case — the same lesson Chapter 22 reached for veers, now for upgrades.

Hosted models drift under you. Chen, Zaharia, and Zou ran a fixed task set against the same hosted model names months apart and found substantial behavioral change on several tasks, some of it regressive (Chen, Zaharia & Zou, 2023). “Every model revision is a new system wearing an old name” is not rhetoric — it is measured, and it applies even when you did not initiate the upgrade. The pinned suite is the only way to notice.

A refreshed suite moves the number on its own. The harness-hash guard catches a template or param change; it does not catch a change in which cases the suite contains. When a coverage review swaps in new fixtures, expect a few points of movement from the re-sampling alone: Recht and colleagues rebuilt standard vision test sets to the original protocol and still saw a 3–15% accuracy drop that was intrinsic to the independent draw, not leakage or overfitting (Chapter 16). A flip on a case that entered in the same refresh is not yet a negative flip — attest the suite’s composition, not just the harness, before reading the diff.

Lab 23: the two-revision diff

PROPOSED, not executed: no author-measured results are reported. The evidence this chapter requires is the reader’s own diff table.

Setup. Take any pinned suite (or assemble one: the 12 refund cases + bundles + deterministic params from Chapters 17–21). Access two revisions (or simulate with two param/template configurations if a second weights revision is unavailable — labeled honestly as a harness diff, not a weights diff).

Task.

1. Write H1/H2/H3 with distinct diff FORECASTs before running rev-B (e.g., “H1: ≥2 pass→fail with harness equal; H2: any flip with harness-hash mismatch voids to UNKNOWN-rerun; H3: flips both directions ≥1 each”).
2. Independent variable: revision only. Controlled variables: suite bytes, harness, params, seeds — all hashed equal first.
3. Run both revisions (≥5 trials/case), record OBSERVATION (per-case counts + diff rows) and UPDATED BELIEF. Declare the gate (green/yellow/red) with the rows cited. Any prompt edits during the comparison restart the diff from zero.
4. File the diff as the rollout record: suite hash, both revision ids, harness hash, per-case rows, gate decision.

Success criterion. A completed per-case diff matching one pre-written pattern plus the filed gate decision. An upgraded endpoint without the diff table is explicitly not completion — deployment is not diagnosis.

Companion tool: Behavioral Diff Explorer

What it accepts: the pinned suite (fixtures + bundles + params + seeds + hashes), the per-case trial series for both revisions, the harness-equality attestation, and the gate policy with FORECASTs. What it performs: it verifies harness equality, renders the per-case diff (unchanged/fixed/broken/new-pass with counts), checks the pattern against FORECASTs, refuses any upgrade verdict on single trials, unequal harnesses, or mid-comparison prompt edits, and stamps the rollout record with all hashes. What it can establish: how behavior moved between these two revisions on this suite — which cases broke, which fixed, under this harness only. What it cannot establish: why the weights moved (no interior claim), whether rev-B is “better” beyond the suite (suite coverage bounds every verdict), or future stability — the next revision re-opens everything. It never treats headline scores, single-case anecdotes, agreement across paraphrases, or silent prompt-repairs as diff evidence. How its output changes your next action: green rolls forward with the diff filed as the new baseline; yellow pins the regressed slice to rev-A and opens a repair branch; red rolls back with the diff as incident artifact — and every outcome routes its broken rows backward into Chapters 18–22 probes against the new revision.

Paper form, sufficient for this chapter:

SUITE: ___ (hash ___)   HARNESS: ___ (hash ___, equal Y/N)   TRIALS: ___/case
rev-A: ___ -> rev-B: ___   BROKEN: ___ (cases ___)   FIXED: ___ (cases ___)
GATE: green / yellow / red (rows: ___)   BASELINE FILED: ___

Where a software implementation does not yet exist in the reader’s stack, this record is the tool. The diff discipline precedes any automation.

Reusable procedure: every upgrade gets this diff

1. Freeze the suite — fixtures, bundles, params, seeds, hashes.
2. Attest harness equality — hash match or stop.
3. Run both revisions — ≥5 trials per case, per-case counts.
4. Read the diff rows — broken vs. fixed vs. unchanged against FORECASTs.
5. Gate and file — green/yellow/red with rows cited; diff becomes the baseline.

Failure modes

• Headline upgrade. Migrating on announced scores without running the reader’s suite. Someone else’s exam certifying your production.
• Harness-drift diff. Comparing across template/param/seed changes and blaming weights. Hash the harness or the diff is fiction.
• Anecdote rollback. Reverting on one ticket or staying on one demo. Both directions count; rows decide.
• Repair-during-compare. Tuning prompts mid-diff. The joint system improves while the weight question stays UNKNOWN — record the weight diff first, repair second.
• Suite rot. A suite that never gains the new failures as cases. Every incident’s broken rows join the suite or the next upgrade re-learns them at ticket cost.
• Revision-label vagueness. “The new model” instead of a pinned revision id. Unnamed revisions cannot be diffed, rolled back to, or held — the rollout record needs ids, not adjectives.
• Gate shopping. Re-running the diff with tweaked params until it turns green. A gate that moves when the result displeases is a ribbon, not a control — pre-write the gate policy with the FORECASTs.

Limits, per contract: one diff compares two revisions on one suite under one harness; it explains nothing interior, certifies nothing beyond suite coverage, and expires at the next revision, template, or param change. UNKNOWN wherever harnesses differ, trials run single, or prompts moved mid-compare.

References

• Mahdi Milani Fard, Quentin Cormier, Kevin Canini, and Maya Gupta. Launch and Iterate: Reducing Prediction Churn. Advances in Neural Information Processing Systems 29 (NeurIPS), 2016. https://papers.nips.cc/paper/6053-launch-and-iterate-reducing-prediction-churn
• Sijie Yan, Yuanjun Xiong, Kaustav Kundu, Shuo Yang, Siqi Deng, Meng Wang, Wei Xia, and Stefano Soatto. Positive-Congruent Training: Towards Regression-Free Model Updates. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021, pp. 14299–14308. https://arxiv.org/abs/2011.09161
• Stella Biderman, Hailey Schoelkopf, Lintang Sutawika, Leo Gao, Jonathan Tow, et al. Lessons from the Trenches on Reproducible Evaluation of Language Models. arXiv:2405.14782, 2024. https://arxiv.org/abs/2405.14782
• Lingjiao Chen, Matei Zaharia, and James Zou. How Is ChatGPT’s Behavior Changing Over Time? arXiv:2307.09009, 2023. https://arxiv.org/abs/2307.09009
• Jessica Echterhoff, Fartash Faghri, Raviteja Vemulapalli, Ting-Yao Hu, Chun-Liang Li, Oncel Tuzel, and Hadi Pouransari. MUSCLE: A Model Update Strategy for Compatible LLM Evolution. Findings of the Association for Computational Linguistics: EMNLP 2024 (arXiv:2407.09435). https://arxiv.org/abs/2407.09435
• Jia Sheng and Yiwei Lu. No Universal Signal Predicts Sample-Level LLM Regression under Version Updates. arXiv:2608.13607, 2026. https://arxiv.org/abs/2608.13607

Debugging Checklist

• Suite pinned (fixtures + bundles + params + seeds + hashes)?
• Harness equality attested by hash before comparing?
• Both revisions run ≥5 trials per case; per-case counts recorded?
• H1/H2/H3 diff FORECASTs written before rev-B ran?
• No prompt edits during the comparison (or diff restarted)?
• Gate declared (green/yellow/red) with diff rows cited?
• Revision ids (not adjectives) recorded for both sides?
• Gate policy pre-written with the FORECASTs (no gate shopping)?
• New broken rows added to the suite for the next upgrade?
• Diff filed as new baseline; broken rows fed back to Ch 18–22 probes?

What This Chapter Established

• Upgrade/diff discipline: pinned behavioral suite, harness-equal both-revision runs, per-case diff table, gated rollout (green/yellow/red) — demonstrated on the rev-A→rev-B refund regression as H1-yellow — constructed illustration, no measured runs claimed.
• Lab 23 as a proposed two-revision diff record the reader executes; the Behavioral Diff Explorer contract (accepts/performs/can-establish/cannot-establish/next-action).
• What was NOT proved: any interior/representational mechanism (behavioral movement only, by design), any beyond-suite “better model” claim, or anything about the next revision.
• Research grounding: equal aggregate accuracy hides per-example flips (“prediction churn” — Milani Fard et al.), which is why the diff is per-case; the BROKEN column is the “negative-flip” set, positive-congruent training reduces it for internally trained models (Yan et al.), and MUSCLE’s compatibility adapter carries that to LLM updates (Echterhoff et al., ~40% fewer negative flips on one Llama update); no interior signal — confidence, logit margin, attention entropy, cross-version divergence — reliably predicts which cases will flip (Sheng & Lu), so the full pinned suite is not optional; harness implementation differences swamp real model differences (Biderman et al. — the harness-hash guard); a refreshed suite moves the number by re-sampling alone (3–15% intrinsic, Recht et al., Ch16 — attest suite composition too); and hosted models drift under a fixed name (Chen, Zaharia & Zou).
• Part IV’s closing map: 17 set the opacity stance, 18 triaged the layer, 19 certified the bytes, 20 ledgered the window, 21 measured the distribution, 22 located veers with weak signals, this chapter made behaviors survive revisions as assets.
• What Part IV never did: open the weights, explain a decision, or certify a model — every verdict stayed at the boundary, and every interior sentence stayed a lead or UNKNOWN.

Next

Part IV ends with its assets filed: fixtures, bundles, ledgers, distributions, diffs — a boundary-certified system whose interior never testified and never needed to. But the failures so far were all found work: given bugs, given fixtures, given suites. Part V turns the debugger around — from finding flaws in the machine’s work to reviewing flaws in work the machine itself produced.

The next chapter opens that inversion: AI as builder, designer, researcher, and reviewer — where the suspect output arrives with a confident author that cannot be interviewed, only audited. None of Part IV’s assets certify that work; they merely equip its interrogation. The interrogation starts with the same first question every part of this book has asked: what was intended, what was observed, and where do they first diverge?