Part V — Debugging AI-Assisted Development and Research

Models are now collaborators — debug their work products

Part IV established behavior-first debugging of opaque models: swap probes that change one input property at a time, input inspection that verifies what the model actually received, and sampling discipline that refuses single-run verdicts. The model was the patient on the table.

That relationship just changed. Models are now collaborators producing code, designs, and research — work products you will merge, ship, and cite. Debug their work products, not just their outputs. An eloquent answer that fails its acceptance criteria is not a curiosity; it is a defective artifact entering your system.

OBSERVATION: the same model asked to write a rate limiter, sketch an API, summarize prior art, and review a pull request produces four artifacts that fail in four unrelated ways — wrong boundary condition, unscalable schema, invented citation, missed race condition. HYPOTHESIS: each role fails differently and leaves different evidence; one generic “check the AI’s work” step cannot catch all four. INFERENCE: none yet — role framing is the claim to test, not the conclusion. This chapter taxonomizes the roles so each gets its own evidence and its own failure signature.

This chapter’s question: when an AI-assisted artifact disappoints, which role produced it — and what evidence does that role owe you?

Generation outruns verification

Part IV’s model was slow to interrogate and fast to run. The collaborator is the opposite: it produces a rate limiter, a schema, a literature review, and a pull-request review in the time it takes to read one of them. Generation capacity has risen sharply. Independent verification capacity — running the tests, pricing the constraints, resolving the citations, reading the diff line by line — has not.

That gap is the engineering problem, and it is not “AI sometimes makes mistakes.” It is that plausible artifacts accumulate faster than their assumptions, provenance, and behavior can be independently checked. On inspection alone, an unverified artifact that reads well is indistinguishable from a verified one.

This is measured, not speculative. Perry and colleagues had 47 participants complete security-sensitive programming tasks with and without an AI assistant, and the assisted group wrote significantly less secure code — while being more likely to believe their code was secure (Perry et al., 2023). The artifact got better-looking and the verification got worse at the same time. The same inversion shows up on ordinary work: a 2025 randomized trial had 16 experienced open-source developers complete real tasks in repositories they had worked in for years, with and without AI tools. They forecast that AI would speed them up by about a quarter, and afterwards still believed it had — but they were measured roughly 19% slower with AI, the time going into reading, checking, correcting, and integrating the model’s output (Becker et al., 2025). Bound that result: a larger follow-up cohort from the same group found the effect small and not clearly different from zero. The durable finding is not a fixed slowdown — it is the gap between how fast the work felt and how fast it was, and the fact that the effort moved into verification. The only defense is to make each artifact carry the evidence its role owes — and every role makes a claim the artifact cannot settle by itself:

• Builder claims the implementation satisfies the requirement — owes executed tests, runtime behavior, and the diff.
• Designer claims the architecture satisfies its constraints — owes the constraints, priced alternatives, and a tradeoff analysis.
• Researcher claims a proposition is supported — owes the source, the passage, the conditions, and the attribution.
• Reviewer claims the artifact passes evaluation — owes an explicit rubric and independent observations.

AI-generated evidence is not independent evidence: a model’s own test summary, its stated confidence, and a second agreeing generation all come from the process that produced the artifact. Independence means a check the artifact cannot author — an executed test, arithmetic against a pinned bound, a byte retrieved on a separate path.

Three of these roles get dedicated treatment in this Part — Builder (Chapters 25, 26, 29), Designer (Chapter 27), Researcher (Chapter 28). The Reviewer does not, and the omission is deliberate. Debugging an AI reviewer means asking whether its evaluation was itself sound, which needs machinery this book has not built yet: evidence discipline, agent traces, diagnostic hypotheses, independent verification, and a benchmark to score a debugger against. Chapters 44, 49, 50, and 51 take it up once that groundwork exists. The Reviewer is named here; evaluating the evaluator becomes its own debugging object later.

The accumulating record: the AI work-product case file

Chapters 25–29 are not six unrelated checklists. Each adds a section to one diagnosable record of the work product — call it the AI work-product case file:

AI WORK-PRODUCT CASE FILE
  role                  — Ch 24
  intent                — Ch 25
  context / working set  — Ch 26
  assumptions
  design / research evidence — Ch 27–28
  artifact + validation
  first divergence
  decision / route

The trajectory that produced all of it — Chapter 29 — is the last section this Part adds. Later parts extend the file; Chapter 45 develops it into a formal AI crash dump. For now the point is only that a single case about AI work is taking shape, one evidence type at a time, tracing back from the shipped decision through every input that shaped it.

Why “read it over carefully” fails first

The obvious move — a careful read-through by a senior engineer — fails because eloquence defeats inspection. Four defects hide behind fluent prose:

1. Role confusion. The reviewer reads code as prose (style, naming) when the defect is behavioral (off-by-one under load). Each role needs its own checklist, not one general carefulness.
2. Missing oracle. Without acceptance criteria written before generation, “looks right” becomes the standard. Post-hoc review rationalizes whatever was produced.
3. Evidence asymmetry. The builder’s evidence (tests run) differs from the researcher’s (sources retrieved) differs from the reviewer’s (lines actually examined). One review pass checks the wrong evidence for three roles out of four.
4. Explanation-as-trace. Asking the model “are you sure this is correct?” returns a confident paragraph — verbal behavior, not verification (Chapter 3). Agreement sampled twice is not proof.

Overreliance on AI output is a documented effect, and the usual fix does not work: Buçinca and colleagues found that adding explanations to AI suggestions did not reduce people’s tendency to accept wrong ones, and sometimes increased it. What did reduce overreliance were cognitive forcing functions — interventions that make the person do some of the reasoning before seeing or accepting the AI’s answer — though participants rated those designs as more effortful and less pleasant (Buçinca, Malaya & Gajos, 2021). The role → evidence → failure table below is a cognitive forcing function: it makes you commit to what each role owes before you read the fluent artifact. Expect it to feel slower. That is the mechanism, not a side effect.

OPINION: “AI pair programmer” is a job title covering four jobs. Debugging starts by firing the generalist and hiring four specialists — each with its own acceptance evidence.

The mental model: role determines evidence; evidence determines the debug procedure. You do not debug a researcher with a test suite or a builder with a bibliography. Name the role first, demand its evidence second, run its procedure third.

The method: role → evidence → failure

Name the role before opening the artifact. The table is the triage instrument:

Procedure per role:

1. Builder: run the artifact (tests, reproduction script, boundary inputs). First divergence: expected vs. observed execution. Demand the working set (what files the builder saw) before theorizing.
2. Designer: map each claim against constraints (load, latency, consistency, blast radius). First divergence: the constraint row with no supporting measurement or calculation.
3. Researcher: resolve every reference to a retrieved byte (Chapter 3 hygiene). First divergence: the first claim with citation shape but no source hash.
4. Reviewer: diff the review’s coverage against the diff under review (which lines, which paths). First divergence: the unexamined critical path the approval implicitly blessed.

    flowchart TD
    A["AI-generated artifact"] --> RD["declare every role it plays: builder / designer / researcher / reviewer"]
    RD --> EV["demand each role's evidence BEFORE reading the prose"]
    EV --> B["builder: executed tests + input hashes + the diff"]
    EV --> D["designer: constraint map + priced alternatives + tradeoff table"]
    EV --> R["researcher: retrieved bytes + source hashes + timestamps per claim"]
    EV --> V["reviewer: line-anchored findings + what was examined + severity rationale"]
    B --> FD["first divergence: earliest evidence row missing or contradicting intent"]
    D --> FD
    R --> FD
    V --> FD
    FD --> RT["route to the role's procedure: Ch25 intent / Ch26 context / Ch27 design / Ch28 research / Ch29 trajectory"]
  

ROLE TRIAGE (run before any deep debugging):
artifact type: code / design / research / review (circle one)
role evidence present? builder: tests+hashes Y/N | designer: constraints+tradeoffs Y/N
  researcher: sources+hashes Y/N | reviewer: coverage+anchors Y/N
first divergence: the earliest evidence row that is missing or contradicts intent: ___
route: Ch25 (intent) / Ch26 (context) / Ch27 (design) / Ch28 (research) / Ch29 (trajectory)

OBSERVATION (constructed illustration, not a measured run): a generated rate limiter passed style review; the triage row showed builder evidence missing (no test run recorded), designer evidence absent (no burst-load constraint mapped), reviewer evidence thin (approval covered 12 of 47 changed lines). UPDATED BELIEF: role confusion supported for this instance — three roles’ evidence missing, one role’s prose present. The fix is role-specific evidence collection, not more careful reading.

Example: one prompt, four defective work products

A team asks for “a cached user-profile endpoint with review.” The model delivers code, a scaling note, prior-art references, and a self-review — each fluent, each defective in its role-specific way:

• Builder defect: cache TTL refreshes on read, so hot keys never expire — fails only under sustained load, invisible to a read-through. Caught by running a TTL-boundary test, not by reading.
• Designer defect: proposes fan-out reads to three services per request with no latency budget — elegant diagram, p99 violation by construction. Caught by constraint mapping (budget: 200 ms; line items sum to 450 ms), not by admiration.
• Researcher defect: cites two papers supporting the invalidation strategy; one citation resolves to an unrelated paper, the other to no retrievable byte at all. Caught by provenance resolution, not by summary quality.
• Reviewer defect: self-review approves with “handles edge cases” while the TTL path and one error branch were never exercised. Caught by coverage diff (lines examined vs. lines changed), not by tone.

# builder-evidence probe: run the boundary the prose never mentions
import time
cache.set("u:1", profile, ttl=60)
time.sleep(61)  # OBSERVATION point: expired or sticky?
assert cache.get("u:1") is None, "TTL refreshes on read — hot keys never expire"
# HYPOTHESIS H1 (builder defect): read-refresh logic; H2 (test-gap defect): no TTL test run.
# Discriminator: the test above, run 3+ times (sampling discipline) with code+input hashes logged.

No confidence statement, agreement count, or downstream symptom (“latency looks fine in dev”) substitutes for the role’s evidence. The code runs or it does not; the design sums within budget or it does not; the citation resolves or it does not; the review covers the path or it does not.

Research lineage: the bottleneck and the forcing function

The verification gap is empirical. The two poles are instructive. On greenfield, well-scoped work — implement an HTTP server from scratch — a controlled trial found GitHub Copilot users about 56% faster (Peng et al., 2023). On verification-heavy work in systems the developer already owns, the METR trial above found the gain shrink and reverse. In both, the review step is where the time went — it just cost less than it saved in the first case and more in the second. The engineering response is the same either way: attach the missing evidence to the artifact rather than hope the reader supplies it. (One further wrinkle: AI-assisted contributions can move the verification cost onto reviewers and maintainers rather than remove it — early data on “vibe-coded” open-source contributions shows disproportionate review overhead landing on project maintainers.)

Explanations are not verification, and can make overreliance worse. Buçinca and colleagues’ finding — that AI explanations did not curb acceptance of wrong answers — is the human-factors version of Chapter 3’s rule. A model’s rationale for its own code is another fluent output; the thing that reduces error is a forcing function that makes the human engage with the evidence directly.

The Reviewer role is deferred for a reason the literature supports. Panickssery and colleagues showed that LLM evaluators recognize and systematically favor their own generations, so an AI reviewing AI-written code carries a self-preference bias on top of the ordinary judge biases (Panickssery, Bowman & Feng, 2024). Debugging an AI reviewer therefore needs the eval-instrument machinery this book builds later (Chapters 44, 49–51), not a checklist here.

Lab 24: role-swap triage drill (proposed)

PROPOSED, not executed: no author-measured results are reported. The evidence this chapter requires is the reader’s own triage table.

Setup. Take one AI-generated artifact bundle (code + design note + references + review comment) you already have, or generate one fresh with a pinned prompt, model version, and seed. Freeze inputs, artifact bytes, and hashes — the review lens is the independent variable; the artifact is controlled.

Task.

1. Write H1/H2/H3 before triaging: H1 — the defect is builder-class (runnable behavior wrong); H2 — designer-class (constraint violated); H3 — researcher/reviewer-class (provenance or coverage missing). Give each a distinct predicted evidence row (e.g., “H1: TTL-boundary test fails; H2: latency line-items exceed budget; H3: ≥1 citation lacks a source hash”).
2. Fill the role→evidence→failure table: for each role, record evidence present/absent with hashes, and the first divergence row. Run each role’s probe once minimally (execute one boundary test; sum one budget; resolve all citations; diff review coverage) with ≥3 trials where generation is re-sampled.
3. Record OBSERVATION (evidence rows verbatim) and UPDATED BELIEF per hypothesis. Conviction requires the predicted row pattern; any other pattern is UNKNOWN with the next probe named.

Success criterion. A completed triage table routing the defect to exactly one of Chapters 25–29 with the first-divergence row quoted. A “careful read” paragraph without this table is explicitly not completion.

Companion tool: AI Work Product Review Checklist

What it accepts: the artifact bundle, the declared role(s), pinned inputs/hashes, and per-role evidence rows (test logs, constraint maps, source hashes, coverage anchors). What it performs: it enforces role declaration before review, blocks verdicts while any claimed role’s evidence row is empty, checks first-divergence ordering (earliest missing/contradictory row first), and requires repeated trials for re-sampled generations. What it can establish: whether the artifact carries its role’s evidence and which role’s procedure the defect routes to — under the examined revision only. What it cannot establish: correctness beyond the evidence (passing one boundary test does not certify the builder; resolving citations does not verify the synthesis logic), future validity, or model-internal causes. It never treats fluency, confidence, agreement, or single-run success as diagnosis. How its output changes your next action: a missing-evidence row routes to evidence collection (Chapters 25–28); a present-but-contradictory row routes to discriminating intervention; a fully-evidenced artifact routes to targeted debugging (Ch29 for trajectories) or acceptance.

Paper form, sufficient for this chapter:

Role(s) claimed: builder / designer / researcher / reviewer (circle)
Builder: tests run ___ (hash ___) | boundaries covered ___ | result ___
Designer: constraints ___ | budget line-items ___ | rejected alternatives ___
Researcher: claims ___ | resolved to bytes ___ | unresolved ___
Reviewer: lines changed ___ | lines examined ___ | unexamined critical paths ___
FIRST DIVERGENCE: ___   ROUTE TO: Ch25 / Ch26 / Ch27 / Ch28 / Ch29

Where a software implementation does not yet exist in the reader’s stack, this record is the tool. Role declaration precedes all debugging.

Reusable procedure: triage by role before debugging

1. Declare the role — builder, designer, researcher, reviewer (artifacts often mix; list all).
2. Demand the role’s evidence — tests, constraints, sources, coverage — with hashes.
3. Find the first divergence — earliest missing or contradictory evidence row.
4. Route, don’t generalize — send the defect to the role’s chapter procedure.
5. Pin the prevention — the missing evidence row becomes a required checklist item.

Failure modes

• Generic carefulness. One review pass for four evidence types. Eloquence passes; defects ship.
• Post-hoc oracle. Acceptance criteria invented after seeing the artifact. The artifact always passes its own bespoke exam.
• Explanation-as-verification. “The model confirms the design is sound” filed as evidence. Verbal behavior is not a trace (Chapter 3).
• Single-sample acceptance. One good generation proves reliability. Nondeterministic collaborators require repeated trials.
• Multi-variable re-prompting. Changing prompt, context, and seed at once after a failure. Nothing learned; luck bookings as skill.
• Downstream-symptom relief. “Staging looks fine” closing a missing-evidence defect. Symptoms are not coverage.

Limits, per contract: one triage record routes one artifact revision under one input hash; it does not certify the artifact, does not explain model internals, and does not transfer across re-prompts. UNKNOWN where evidence rows are unfillable without the working set (see Chapter 26).

References

• Neil Perry, Megha Srivastava, Deepak Kumar, and Dan Boneh. Do Users Write More Insecure Code with AI Assistants? Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security (CCS), 2023, pp. 2785–2799. https://doi.org/10.1145/3576915.3623157
• Joel Becker, Nate Rush, Elizabeth Barnes, and David Rein. Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity. arXiv:2507.09089, 2025. https://arxiv.org/abs/2507.09089
• Sida Peng, Eirini Kalliamvakou, Peter Cihon, and Mert Demirer. The Impact of AI on Developer Productivity: Evidence from GitHub Copilot. arXiv:2302.06590, 2023. https://arxiv.org/abs/2302.06590
• Zana Buçinca, Maja Barbara Malaya, and Krzysztof Z. Gajos. To Trust or to Think: Cognitive Forcing Functions Can Reduce Overreliance on AI in AI-Assisted Decision-Making. Proceedings of the ACM on Human-Computer Interaction 5 (CSCW1), 2021, article 188. https://doi.org/10.1145/3449287
• Arjun Panickssery, Samuel R. Bowman, and Shi Feng. LLM Evaluators Recognize and Favor Their Own Generations. Advances in Neural Information Processing Systems 37 (NeurIPS), 2024. https://arxiv.org/abs/2404.13076

Debugging Checklist

• Role(s) declared before review (builder/designer/researcher/reviewer)?
• Per-role evidence demanded with hashes (tests/constraints/sources/coverage)?
• First divergence row identified (earliest missing/contradictory evidence)?
• Defect routed to the role’s chapter (25/26/27/28/29), not generically “fixed”?
• No explanation, confidence, agreement, or single-run result cited as verification?
• Repeated trials where generation was re-sampled?
• Missing evidence row converted into a required checklist item?

What This Chapter Established

• Role framing for AI collaborators: builder, designer, researcher, reviewer — each with distinct evidence and distinct failure signatures, summarized in the role→evidence→failure table.
• The verification bottleneck as the Part’s motive: generation capacity has outrun independent-verification capacity, so plausible artifacts accumulate faster than their assumptions, provenance, and behavior can be checked — and AI-generated evidence is not independent evidence. Measured twice as a perception–reality inversion: AI assistance produced less secure code and more misplaced confidence (Perry et al.); experienced developers felt faster with AI on their own repos while being measured slower, the effort shifting into verification (Becker et al., with the slowdown magnitude itself contested by a later cohort). AI explanations do not curb overreliance while cognitive forcing functions do (Buçinca et al.), and the role table is one.
• The AI work-product case file introduced as the record Chapters 25–29 extend section by section (role → intent → context → design/research evidence → artifact/validation → first divergence → route); a precursor to the Chapter 45 crash dump, not that dump.
• The role-triage method (declare → demand evidence → first divergence → route) demonstrated on a constructed four-defect bundle; no measured runs claimed.
• Lab 24 as a proposed triage drill the reader executes; the AI Work Product Review Checklist contract (accepts/performs/can-establish/cannot-establish/next-action).
• What was NOT proved: any claim about model capability, any causal account of why a role failed, or any certification of an artifact. Triage routes; it does not convict.

Next

Triage keeps routing defects to the same upstream address: the intent was never written down. The builder built the wrong thing correctly, the designer optimized unstated goals, the researcher answered an unasked question. Chapter 25, “Debugging Intent,” makes intent a debuggable artifact — acceptance criteria before generation, and the intent-diff that separates “wrong output” from “underspecified input.”