# Appendix A: PyTorch Diagnostic Field Guide
# Book: PyTorch From First Principles
# Chapter URL: https://programmer.ie/books/pytorch-from-first-principles/appendix-a/
# Prose fingerprint (sha256, normalized): 459a6efbefd37e93ce3a1525215c9d9bca15708d41403dc4447e42730fa2ea95
# Snapshot generated at build time by Hugo. This is the complete rendered prose
# of the chapter; it is not a transcript of the source file.

This appendix introduces no new PyTorch mechanism.
Everything here was earned earlier in the book by building something small, breaking it deliberately, measuring what changed, and locating the first place where reality stopped matching the intended computation.
The purpose of this appendix is different.
When a real model is failing, you usually do not need another explanation of autograd, broadcasting, attention or CUDA. You need to answer a narrower question:
Which instrument should I reach for first?
The book’s recurring move was:
predict ↓ observe ↓ find the first divergence ↓ identify the mechanism ↓ change one thing ↓ verify the repair That is the default procedure. The sections below route common symptoms to the smallest instrument that can produce useful evidence.
The central rule remains:
Make hidden structure visible before guessing.
Start here: route the symptom What you see First question First instrument Chapter A tensor error appears far downstream Where did the tensor first become wrong? tensor trace / explicit shape contract 2 grad is missing Where does the path from loss to parameter first disappear? autograd reachability / graph inspection 3 Gradients exist but the raw network still fails Which claim is actually unsupported? six-claim audit 4 A layer exists in forward() but disappears from checkpoints or optimization Which structure can see this object? four-structure audit 5 GPU utilization is low or batches arrive irregularly Is the consumer actually waiting for input? producer-consumer split 6 Training succeeds but the model is seeing the wrong representation At which transform boundary did meaning change? one-sample stage trace + semantic probes 7 CNN dimensions stop matching What shape should this layer have produced? derive-then-trace shape ledger 8 A linear layer returns too many scores, or a feature-space number is being misread What does one vector represent, and what quantity are we interpreting? feature-space inspector 9 Attention has the right shape but behaves incorrectly Which correctness level fails first? attention ledger + intervention 10 Forward, backward and optimizer.step() all run, but the model does not learn Which expected consequence of learning is first missing? learning ledger 11 The program is correct but slow or memory-hungry Which resource is limiting the workload? measurement contract + performance ledger 12 torch.compile is correct but recompiles, pauses or loses its speedup Which compiler assumption stopped holding? graph-break / guard / recompile evidence 13 A new run looks worse than an old run Is there a difference worth explaining? baseline variation + paired comparison + run record 14 A complete transformer behaves strangely Which subsystem owns this symptom? route to the appropriate instrument above 15 Do not interpret this table as a strict sequence that every bug must follow.
It is a routing table.
A single failure can cross several systems. A transform bug may first appear as a training failure. An optimizer-ownership bug may first appear as a flat loss. A compiler problem may first appear as a performance regression.
The job is to move from the visible symptom to the earliest unsupported claim.
The minimum evidence packet Before changing code, capture enough state that the failure can survive your first attempted fix.
For a training problem, the minimum useful packet is often:
one known batch input shape / dtype / device / range target shape / dtype / meaning model mode loss value loss.requires_grad loss.grad_fn expected trainable parameter names grad state for those parameters optimizer parameter membership parameter deltas after one controlled step current learning rate for every parameter group For a performance problem:
workload input shapes dtype device train/eval/inference mode eager/compiled metric warmup measurement window repetitions synchronization boundary peak allocated memory peak reserved memory For a regression:
code version configuration data version seed environment evaluation contract baseline variation paired deltas if pairing is valid The exact packet changes with the question.
The principle does not:
Preserve observations before your intervention destroys the failing state.
1. The training loop: separate forward, backward and update When the system is still small enough, begin with the decomposition from Chapter 1:
forward compute prediction and loss backward compute derivatives and accumulate them into .grad update change parameter values These are separate mechanisms.
loss.backward() does not update parameters.
An update does not clear gradients.
A falling loss does not prove that the intended task is being learned.
Ask What value produced this loss? What gradient was produced by this loss? What parameter value existed before the update? What parameter value exists after the update? Was the old gradient cleared before the next backward pass? Is this still the same parameter object? High-value measurements before = p.detach().clone() loss.backward() grad = None if p.grad is None else p.grad.detach().clone() optimizer.step() delta = p.detach() - before For a controlled plain-SGD step with no momentum or weight decay:
delta ≈ -learning_rate * gradient If that equality fails under the deliberately restricted experiment, the update mechanism is not the one you think it is.
If it holds, do not generalize it blindly to Adam, momentum, weight decay or other stateful update rules.
Stop when You can separately prove:
the forward calculation is the intended one backward produced the expected derivative the intended parameter object moved old gradients are not contaminating the next step Then move to the next layer of the system.
2. Tensors: find the first wrong tensor, not the first illegal one A tensor is not only values.
The useful description is:
shape dtype device stride contiguity axis meaning value meaning The first five are inspectable from the tensor.
The last two belong to the program’s contract.
First move Write the expected tensor before running the operation.
Not:
I expect something like [32, ?] but:
[B, T, D] B = examples T = positions D = feature coordinates Then compare expectation with observation at each boundary.
Use deterministic values when layout is the problem Random numbers hide rearrangements.
For reshape, transpose, head splitting or permutation bugs, use:
x = torch.arange(...) small enough to print.
If two operations produce the same shape, inspect where the values moved.
Common false repairs reshape until the exception disappears change in_features to match the tensor that arrived change in_channels to match an NHWC batch squeeze dimensions without naming what they represent These can turn an upstream mistake into a legal downstream architecture.
Stop when You have found the first boundary where observed tensor facts or axis semantics stop matching the expected contract.
Repair there.
3. Autograd: find where the dependency path stops existing When a parameter does not receive the expected gradient, do not begin with the final .grad.
Start from the question:
Does this loss actually depend on this parameter in the graph that executed?
Autograd records the operations that ran during this forward pass. It does not record the model you intended to run.
Evidence ladder parameter.requires_grad parameter.is_leaf loss.requires_grad loss.grad_fn parameter.grad after a controlled backward pass reachability from loss back to expected leaves A healthy loss.grad_fn proves that the loss has some differentiable history.
It does not prove that every intended parameter lies on that history.
Distinguish grad is None from:
grad is a tensor of zeros Under a controlled pass that begins with .grad = None:
None means no gradient was accumulated into that parameter; a zero tensor means autograd materialized a gradient whose accumulated derivative is zero. Those are different mechanisms.
Common causes of a missing path detach() torch.no_grad() around part of the forward computation a branch that did not execute a parameter that was not used rebinding to a new non-tracked tensor Stop when You can point to the first operation or branch after which the intended dependency no longer exists.
4. A raw neural network: ask which of six claims is failing Chapter 4 established that “the gradients look fine” answers only one question.
For a network that is mechanically running, classify the failure across six claims:
Claim Question Useful evidence Representation Can this architecture express the required function? algebra, a falsifying probe, simpler baseline Computation Is the forward pass computing the architecture intended? intermediate tensors, shape/value comparison, independent implementation Objective Does the loss encode the intended problem? manual loss check, trusted reference, logits/target contract Differentiation Does each intended parameter have a path to the loss? autograd reachability, .grad Update Do the intended parameters actually change? parameter snapshots and deltas Evaluation Are we measuring the behavior we actually care about? held-out data, meaningful baseline Why this matters These failures can look similar:
missing nonlinearity symmetric initialization zero initialization detached parameter healthy gradient on an un-updated parameter wrong objective No single gradient statistic distinguishes all of them.
Stop when You can say which claim the evidence supports and which claim is still unsupported.
Do not use evidence for one claim as proof of another.
5. nn.Module: compare the four structures When an object seems to disappear, stop thinking of “the model” as one thing.
There are four overlapping structures:
1. Python object graph 2. registered module tree 3. autograd graph 4. optimizer parameter groups They answer different questions.
Operation Structure it consults forward() through normal Python attribute access Python objects named_modules() registered module tree named_parameters() registered parameters named_buffers() registered buffers state_dict() registered parameters + persistent buffers .to(device) registered parameters + buffers train() / eval() registered modules loss.backward() autograd graph created by execution optimizer.step() that optimizer’s parameter groups The diagnostic question Which structure does the failing operation traverse, and is my object actually in it?
A tensor can:
require gradients receive gradients participate in forward and still not be:
registered saved moved by model.to(...) owned by the optimizer Object identity matters If a head is replaced after optimizer construction, the optimizer may still own the old parameter objects.
Names and shapes can look correct.
The identities are wrong.
Stop when You can reconcile:
what Python can reach what PyTorch registered what autograd reached what the optimizer owns for every object relevant to the failure.
6. DataLoader and input delivery: split producer from consumer “Low GPU utilization” is not a DataLoader diagnosis.
First ask whether the training loop is waiting for batches.
The smallest useful measurement is:
t0 = perf_counter() batch = next(iterator) t1 = perf_counter() train_step(batch) t2 = perf_counter() Now you have two different observations:
wait = t1 - t0 work = t2 - t1 Model the pipeline producers ↓ queue / buffering ↓ consumer Then separate:
producer throughput consumer throughput buffering end-to-end throughput More workers can overlap waiting.
They cannot create more of a CPU resource that is already saturated.
Record tails, not only averages For batch wait, capture:
median p95 maximum A low mean with a large tail can still starve the consumer repeatedly.
Sweep one control at a time For example:
num_workers = 0, 1, 2, 4, 8 recording:
examples/s batch-wait distribution post-batch work time If wait falls while consumer work rises and end-to-end throughput is unchanged, the workers are competing with the consumer for the same resource.
Stop when You can name the actual limiting resource or phase:
read decode transform collate process startup CPU contention host-to-device transfer consumer compute Then repair that mechanism.
7. Transforms and preprocessing: trace one known sample A preprocessing pipeline can produce a legal tensor whose meaning is wrong.
Chapter 7 separated the input contract into five categories:
1. STRUCTURE 2. NUMERIC INTERPRETATION 3. SEMANTICS 4. RELATIONSHIPS 5. PROVENANCE They need different instruments.
Category Typical evidence Structure type, rank, shape Numeric interpretation dtype, range, scale, normalization state Semantics known-sample probe, direct inspection Relationships image-mask/box alignment, input-target assertion Provenance recorded split, fitted statistics, checkpoint/preprocessing version The method Find the first transform boundary where the sample stops satisfying the intended input contract.
Start with:
one sample one process no batching controlled randomness Then walk stage by stage.
Important limitation An all-green metadata report is not a clean bill of health.
A tensor cannot tell you:
its RGB channels were interpreted as BGR the augmentation invalidated the label the mask no longer aligns with the image normalization statistics were fitted on validation data the serving tokenizer differs from training Those need explicit probes or records.
Stop when You have identified the earliest stage where one of the five contract categories fails.
Only then reintroduce batching, workers and performance complexity.
8. CNN geometry: derive before execution For convolutional networks, keep two questions separate:
CHANNEL QUESTION C_in → C_out SPATIAL QUESTION H_in,W_in → H_out,W_out C_out is declared.
Spatial size is derived.
The method Derive the expected shape before the layer runs. Compare it with the observed shape. Investigate the first disagreement.
Use a ledger:
stage operation observed input expected output observed output first divergence Never repair the final Linear first For:
mat1 and mat2 shapes cannot be multiplied derive backward:
Where did the flattened feature count come from? Which earlier layer changed C, H or W? Was that change intended? Changing in_features is correct only if the upstream geometry change was intentional.
Remember the semantic limit A shape ledger proves composition.
It does not prove that height and width were not swapped, that preprocessing preserved meaning, or that the architecture is suitable for the task.
Stop when Derived and observed geometry agree at every shape-changing operation up to the failing boundary.
9. Feature space: name the represented object before interpreting numbers When a tensor is shaped:
[B, T, D] ask:
What does one D-vector represent? How many such vectors are present? Which axis contains its coordinates? nn.Linear(D_in, D_out) transforms the last axis and preserves the leading axes.
So [B,T,D] → [B,T,1] means one score per (example, position), not one score per example.
Before interpreting a scalar, name it dot product cosine similarity Euclidean distance classifier score signed distance to a hyperplane They are not interchangeable.
Check invariances Ask what should remain unchanged when:
one vector is rescaled w and b are scaled together one feature coordinate changes units If the quantity changes under something that should not matter to the interpretation, you are probably naming the quantity incorrectly.
Inspector For a representation, useful rows include:
vector norms per-coordinate scales non-finite values pairwise cosine collapse toward zero collapse toward one direction dominant coordinates Stop when You can state:
what one vector represents which axis holds its coordinates what operation acts on those coordinates what numerical quantity came out which invariances that quantity has 10. Attention: four levels of correctness Attention needs more than a shape check.
Use four levels:
LEVEL 1 SHAPE Does the tensor have the dimensions I derived? LEVEL 2 AXIS SEMANTICS Do those dimensions contain the objects I intended? LEVEL 3 NUMERICAL INVARIANTS Do valid query rows normalize over keys? Are forbidden weights zero? Are values finite? LEVEL 4 BEHAVIOR Does a future-token intervention leave the prefix unchanged? Does the implementation agree with a trusted reference? The method derive the axes ↓ execute one stage ↓ verify the invariant introduced by that stage ↓ stop at the first failure High-value checks merge(split(x)) == x scores shape == [B, Nh, Tq, Tk] softmax normalizes over Tk forbidden weights == 0 no fully blocked query unless explicitly supported merged layout restores [B, Tq, E] future-token intervention preserves the causal prefix within tolerance Why Level 4 exists Missing 1/sqrt(Dh) scaling can preserve:
shape axis semantics finite values row sums and still disagree materially with the intended attention computation.
Stop when You know the earliest correctness level and stage that fails.
11. Training that runs but does not learn: walk the learning chain The learning chain is:
TASK ↓ OBJECTIVE ↓ DEPENDENCY ↓ GRADIENT ↓ OPTIMIZER OWNERSHIP ↓ PARAMETER UPDATE ↓ CAPABILITY This is the right tool when:
forward runs loss is finite backward runs gradients exist optimizer.step() runs and the model is still useless What each link asks TASK
Does the target still describe the input? OBJECTIVE
Are the outputs in the representation the loss expects? Can one example reproduce the loss manually? DEPENDENCY
Does the loss depend on the intended parameters? GRADIENT
Are expected gradients present? None, zero, finite, non-finite? OPTIMIZER OWNERSHIP
Does the optimizer own those exact parameter objects? Does it own stale objects that are no longer in the model? PARAMETER UPDATE
Did the intended objects actually move? CAPABILITY
Can the controlled system overfit a tiny fixed batch? Important warning The learning ledger is a state-mutating probe if it calls backward() and optimizer.step().
Run it on a disposable model/checkpoint or deliberately accept the sacrificial step.
Stop when You find the first expected consequence that fails to appear.
Debug there.
Do not tune learning rate, architecture, precision or regularization while an earlier link remains unsupported.
12. Performance: define the comparison before optimizing Performance is not a property of one line of code.
It is a result under a contract:
workload hardware software/runtime execution mode metric measurement boundary First choose the question latency throughput peak memory cold-start cost steady-state cost These can move in opposite directions.
The method Measure → localize → intervene → remeasure.
Performance ledger Capture:
WORKLOAD model / operation input shape dtype device mode eager / compiled MEASUREMENT warmup repeats synchronization metric TIME input wait transfer forward loss backward optimizer logging / synchronization MEMORY allocated reserved peak allocated peak reserved CUDA timing If the device is asynchronous, host wall-clock timing without a completion boundary can measure submission rather than completed device work.
Know whether you are measuring:
host-visible latency device elapsed time end-to-end workload time Memory Keep separate:
allocated live PyTorch-managed allocations reserved allocator-managed pool peak transient high-water mark If memory_allocated() grows across otherwise-identical iterations, the set of live PyTorch allocations is growing. Trace ownership.
Do not call high reserved memory alone a live-tensor leak.
Profiler rule The profiler is for localization.
The unprofiled benchmark is for the final performance comparison.
Stop when You can state:
the metric that matters the phase/resource that dominates it the mechanism hypothesis the one intervention that tests that hypothesis the correctness property that must remain unchanged the uninstrumented before/after result 13. torch.compile: inspect the compiler’s hidden state A compiled call that returns the right number has proved correctness.
It has not proved:
the hot region was captured the graph was reused guards remained valid recompilation stopped steady-state performance improved cold cost amortized A useful progression is:
RUNS ↓ CORRECT ↓ CAPTURED ↓ STABLE ↓ WORTH IT Begin with eager If eager is already wrong, compilation is not the first problem.
Then compare correctness Use a stated tolerance appropriate to the workload.
Then inspect capture Useful evidence can include:
graph-break logs fullgraph=True as a capture test for the exact call structured compiler summaries No one instrument is automatically ground truth in every version. Triangulate when tools disagree.
Then inspect stability Capture recompilation evidence.
Ask:
Which guard failed? shape? dtype? device? Python value? Did the workload legitimately vary on that dimension? Did a cached specialization get reused? Did a new specialization get created? Did the cache limit force an unmatched call to eager? Then ask whether it was worth it Separate:
cold first call warm-cache first call steady-state call recurring recompilation Compute amortization when startup cost matters.
Stop when You can name the compiler assumption that stopped holding, or establish that compilation is stable and the remaining problem is ordinary workload economics.
14. Regressions: establish the difference before explaining it A new metric is worse.
That is an observation.
It is not yet a regression mechanism.
The first three possibilities NOTHING MEANINGFUL CHANGED ordinary run-to-run variation MEASUREMENT CHANGED different evaluation path / split / preprocessing / mode SYSTEM CHANGED code, configuration, data, environment, compiler state Rule out the first two before explaining the third.
Baseline variation Run the unchanged configuration across several seeds.
Report a distribution, not only one number.
The observed min/max spread is context.
It is not a universal statistical detection threshold.
Pair when pairing is valid Shared seeds can make a subtle comparison more sensitive when the two runs preserve enough shared randomness for the seed effects to cancel.
Inspect:
delta(seed_i) = B(seed_i) - A(seed_i) rather than only comparing two independent ranges.
Verify the evaluation contract Record enough to answer:
same validation examples? same preprocessing? same aggregation? same model.eval() state? same checkpoint-selection rule? A fingerprint proves only the fields included in the fingerprint.
Record the run At minimum:
code version config data identity seed environment evaluation contract compiler state if relevant result Bisect carefully Bisection assumes the good/bad predicate behaves suitably.
Changes can interact.
After a boundary implicates one change, apply that candidate alone and confirm the effect.
Stop when Repeated evidence establishes a difference worth explaining and the measurement contract is known to be comparable.
Only then investigate the mechanism.
15. The complete model: route, do not invent a new instrument The capstone’s lesson is that no single diagnostic certifies a large model.
The complete system contains all the earlier systems at once:
text ↓ tokenization ↓ batch/target alignment ↓ embeddings ↓ attention ↓ residual blocks ↓ logits ↓ objective ↓ autograd ↓ optimizer ↓ performance / compilation ↓ generation ↓ evaluation and comparison When the full model fails, route the symptom.
Examples:
Generated text is nonsense but training loss is excellent → TASK / target alignment first A future token changes an earlier logit → attention Level 4 intervention Initial loss is wildly inconsistent with the declared initialization → OBJECTIVE / initialization contract Gradients are finite but the head never moves → optimizer ownership + parameter delta Training is correct but throughput is low → performance ledger Compiled model pauses on new sequence lengths → guards / recompilation New configuration improves one run by 0.001 → baseline variation + paired comparison A large model does not make the earlier instruments obsolete.
It makes choosing the correct one more important.
Symptom-to-evidence field table This table is intentionally redundant with the chapter-by-chapter guide. Use it from the other direction: start from what you observe.
Symptom Tempting explanation Evidence to collect first mat1 and mat2 shapes cannot be multiplied “The Linear layer has the wrong in_features.” Derive every upstream shape and find the first geometry divergence. expected ... channels, got ... “Conv2d.in_channels is wrong.” Name the incoming axes. Check NHWC versus NCHW before editing the layer. .grad is None “requires_grad was forgotten.” Start with grad=None, run one backward pass, then inspect graph reachability. .grad is finite but parameter is unchanged “The learning rate is too small.” Check optimizer object identity and the measured parameter delta. Loss falls but task behavior is wrong “The model needs more capacity.” Verify input-target alignment and the objective on a known sample. Tiny-batch overfit passes “The whole training system is correct.” Remember what it does not prove: task semantics, validation contract, generalization. GPU utilization is low “Increase num_workers.” Measure input wait, transfer, forward, backward and optimizer time separately. nvidia-smi shows high memory “There is a memory leak.” Compare allocated, reserved and peak across repeated fixed-shape iterations. empty_cache() lowers visible memory “The leak is fixed.” Check whether live allocated memory changed. Attention rows have the right shape “Attention is correct.” Check head round-trip, key-axis normalization, forbidden weights and intervention behavior. torch.compile returns correct output “Compilation worked.” Measure capture, guards, reuse/recompilation and workload economics separately. Compiled code is slower “torch.compile is bad for this model.” Separate cold cost, warm cost, recurring recompilation and steady state. Validation changed between commits “The code change caused it.” Establish baseline variation and verify the evaluation contract first. Same seed gives a different run “PyTorch ignored the seed.” Compare thread/device/version/RNG sources and deterministic configuration. Two runs end at the same metric “They are equivalent.” Compare trajectories and the predeclared checkpoint/evaluation policy. The one-bug rule When you have several failures at once, repair one boundary and rerun.
Do not fix:
input layout layer dimensions learning rate optimizer normalization in one edit.
A multi-change repair destroys causal information.
The book repeatedly used the sequence:
find first divergence fix one thing rerun allow the next divergence to reveal itself This is slower per edit and faster per diagnosis.
Observation and intervention are different operations Instrumentation can change the system.
Examples:
hooks can alter lifetime and add overhead profiler instrumentation changes timing .item() can synchronize CUDA printing values inside compiled code can create graph breaks a diagnostic optimizer step mutates optimizer state switching train/eval changes module behavior seeding changes stochastic execution gradient clipping changes the gradient you were trying to inspect Before calling something an observation, ask:
What did this instrument change merely by being present?
If the answer matters, run the final verification again without the instrument.
Known-good references are disproportionately valuable When an operation is difficult to reason about, reduce the argument.
Use:
manual arithmetic versus PyTorch manual loss versus F.cross_entropy manual attention versus scaled_dot_product_attention eager versus compiled healthy preprocessing versus candidate preprocessing healthy run versus candidate run at shared seeds A reference comparison is strongest when every uncontrolled variable is held fixed.
When the outputs disagree, compare intermediate stages in order.
The first disagreement is usually more useful than the final error magnitude.
The smallest experiment that can distinguish two explanations A good debugging experiment does not merely make one hypothesis look plausible.
It makes two explanations predict different outcomes.
Examples:
Missing gradient Hypotheses:
A. graph path is broken B. path exists but local derivative is zero Experiment:
clear grad to None run one backward pass inspect whether grad remains None or becomes a zero tensor Low GPU utilization Hypotheses:
A. input pipeline is starving the device B. model/host execution is the bottleneck Experiment:
measure batch wait separately from post-batch work Compiled slowdown Hypotheses:
A. steady-state kernel is slower B. recurring recompilation dominates Experiment:
measure a repeated fixed-shape steady state capture recompile logs on the real varying workload Apparent regression Hypotheses:
A. model behavior changed B. ordinary seed variation explains the observation Experiment:
run both versions on shared seeds and inspect paired deltas The best experiment is usually not the one with the most instrumentation.
It is the one whose outcomes force the hypotheses apart.
A reusable incident worksheet Copy this into an issue, notebook or debugging session.
PYTORCH INCIDENT SYMPTOM What exactly was observed? EXPECTED What specific value, shape, behavior or metric was expected? REPRODUCTION What is the smallest input / batch / configuration that reproduces it? FIRST CONTRACT Which contract should be checked first? tensor graph ownership input representation geometry attention learning chain performance compiler run comparison OBSERVATIONS What did the instrument actually report? FIRST DIVERGENCE Where does observed behavior first disagree with the expectation? HYPOTHESES What are the smallest two or three mechanisms consistent with that evidence? DISTINGUISHING EXPERIMENT What one experiment makes those hypotheses predict different outcomes? INTERVENTION Change exactly one mechanism. MUST CHANGE Which measurement should change if the diagnosis is correct? MUST NOT CHANGE Which correctness property must remain unchanged? VERIFICATION Did the original failure disappear for the predicted reason? REGRESSION TEST What small test should remain so this exact failure cannot silently return? If you cannot fill in MUST CHANGE and MUST NOT CHANGE, the proposed repair is probably still too vague.
Using AI as a diagnostic partner The most useful general prompt in this book is not “fix my PyTorch model.”
Use something closer to:
Here is a failing PyTorch system. Do not rewrite it and do not propose fixes yet. 1. State the expected contract at each relevant boundary. 2. Tell me which facts I can inspect directly and which require a semantic, behavioral or provenance probe. 3. Identify the smallest instrument from this list that can localize the failure: tensor trace autograd reachability six-claim audit four-structure audit producer-consumer timing preprocessing stage trace CNN shape ledger feature-space inspector attention four-level check learning ledger performance ledger compiler guard/recompile evidence paired run comparison 4. Ask for the missing evidence. 5. Identify the FIRST boundary whose evidence contradicts the intended system. 6. Give at most three mechanisms consistent with that boundary. 7. Propose one experiment that distinguishes those mechanisms. 8. State what should change if the diagnosis is correct. 9. State what must remain unchanged. Do not propose a repair until the first divergence is supported by evidence. The purpose of the prompt is not to make the assistant cautious.
It is to make the assistant falsifiable.
When to escalate Start cheap.
Escalate only when the cheaper instrument passes.
print / assert tensor facts ↓ known-sample contract ↓ graph / ownership inspection ↓ controlled one-step probe ↓ tiny-batch capability test ↓ hooks / detailed activation inspection ↓ profiler / allocator snapshot / compiler logs ↓ multi-seed comparison This is not a universal cost ordering. A seed sweep can be cheap on a tiny workload and a profiler can be cheap on one step.
The principle is:
Use the least invasive instrument that can distinguish the current hypotheses.
When to stop debugging Stop when all four statements are true:
1. You can name the mechanism that produced the symptom. 2. The repair changed the measurement the mechanism predicted. 3. An independent correctness property remained unchanged. 4. The failure can now be expressed as a test, assertion, contract or recorded comparison that would catch it if it returned. A model that “seems to work now” has not met that standard.
A diagnosis has.
The complete diagnostic stack The book began with one scalar parameter and ended with a model too large to hold in your head. The instruments accumulated in the same order.
TRAINING LOOP forward / backward / update TENSOR first wrong tensor AUTOGRAD first broken dependency edge RAW NETWORK representation / computation / objective / differentiation / update / evaluation MODULE STATE Python graph / registered tree / autograd graph / optimizer groups INPUT DELIVERY producer / queue / consumer PREPROCESSING structure / numeric interpretation / semantics / relationships / provenance CNN GEOMETRY derive / trace / first shape divergence FEATURE SPACE represented object / feature axis / quantity / invariance ATTENTION shape / axis semantics / numerical invariants / behavior LEARNING TASK → OBJECTIVE → DEPENDENCY → GRADIENT → OPTIMIZER OWNERSHIP → PARAMETER UPDATE → CAPABILITY PERFORMANCE measurement contract → phase/resource → intervention → uninstrumented remeasurement COMPILATION RUNS → CORRECT → CAPTURED → STABLE → WORTH IT REGRESSION establish the difference → verify the measurement → localize what changed CAPSTONE choose the instrument that owns the symptom There is no final universal checker at the bottom of that list.
That is the point.
Different failures violate different contracts, and the dangerous ones often preserve every simpler contract below them.
A tensor can have the right shape and wrong meaning.
An attention block can preserve every external dimension and compare the wrong objects.
A parameter can have a healthy gradient and never move.
A model can minimize the loss and learn the wrong task.
A benchmark can return a precise number for the wrong measurement.
A compiled model can be numerically correct and catastrophically slower.
A new run can be worse without anything meaningful having regressed.
The common skill is not memorizing the fix.
It is knowing what evidence the current symptom does not yet give you.
Make the hidden structure visible. Find the first divergence. Repair the mechanism that caused it. Verify that the system actually improved.