← PyTorch From First Principles

附录 A:PyTorch 诊断野战指南

把真实 PyTorch 症状路由到能解决它的最小诊断仪器:野战表、证据包、事故工作单、AI prompt、升级规则,以及从张量跟踪到回归、编译执行的完整栈。

这个附录不引入新 PyTorch 机制。

这里的一切都是本书前面挣来的:建小东西,故意弄坏,测变了什么,定位现实与意图算子第一次分手处。

这个附录的目的不同。

真模型挂时,通常不需要另一篇 autograd、广播、注意力、CUDA 讲解。需要回答更窄的问题:

先伸手拿哪台仪器?

本书反复的动作是:

预测
    ↓
观察
    ↓
找到第一个分歧
    ↓
确认机制
    ↓
只改一处
    ↓
验证修复

那是默认流程。下面按常见症状路由到产出有用证据的最小仪器。

中心规则不变:

猜之前先让隐藏结构可见。


从这里开始:症状路由

你看到的 先问 第一台仪器 章
张量错出现在下游很远 张量第一次在哪错了? 张量跟踪 / 显式形状契约 2
grad 缺失 损失到参数的通路第一次消失在哪? autograd 可达 / 图检查 3
梯度在,raw 网还挂 哪个断言真没支撑? 六断言审计 4
层在 forward() 里,检查点优化里消失 哪套结构看得见这个对象? 四结构审计 5
GPU 利用率低或 batch 到得不规律 消费者真在等输入吗? 生产者-消费者拆分 6
训成了,模型看到的表示错了 哪个变换边界含义变了? 一样本阶段跟踪 + 语义探针 7
CNN 维度对不上了 这层该产出什么形状? 推导再跟踪形状台账 8
线性层分数太多,或特征空间数被误读 一向量表示什么,解释的是哪个量? 特征空间检查器 9
注意力形状对行为错 哪个正确性层先挂? 注意力台账 + 干预 10
前向反向 optimizer.step() 全跑,模型不学 学习哪个期待后果先缺席? 学习台账 11
程序对但慢或内存多 哪个资源 limiting workload? 测量契约 + 性能台账 12
torch.compile 对但重编、停顿、加速没了 编译器哪个假设不再成立? 图断裂 / 守卫 / 重编译证据 13
新跑看着比旧跑差 有值得解释的差吗? 基线 variation + 配对比较 + run 记录 14
完整 Transformer 行为怪 症状归哪个子系统? 路由到上面恰当仪器 15

别把这张表当每 bug 必走的严格序列。

它是路由表。

一失败能跨几系统。变换 bug 先表现为训练失败。优化器归属 bug 先表现为平损失。编译器问题先表现为性能回归。

工作是从可见症状走到最早无支撑的断言。


最小证据包

改代码前,捕获足够 state,让失败在第一次试图修复后活下来。

训练问题,最小有用包常常是:

one known batch
input shape / dtype / device / range
target shape / dtype / meaning
model mode
loss value
loss.requires_grad
loss.grad_fn
expected trainable parameter names
grad state for those parameters
optimizer parameter membership
parameter deltas after one controlled step
current learning rate for every parameter group

性能问题:

workload
input shapes
dtype
device
train/eval/inference mode
eager/compiled
metric
warmup
measurement window
repetitions
synchronization boundary
peak allocated memory
peak reserved memory

回归:

code version
configuration
data version
seed
environment
evaluation contract
baseline variation
paired deltas if pairing is valid

exact 包随问题变。

原则不变:

干预毁掉失败 state 前保留观测。


1. 训练循环:分开前向、反向、更新

系统还小,先用第 1 章的分解:

forward     compute prediction and loss
backward    compute derivatives and accumulate them into .grad
update      change parameter values

三独立机制。

loss.backward() 不更新参数。

更新不清梯度。

下降的损失不证明意图任务在学。

问

What value produced this loss?
What gradient was produced by this loss?
What parameter value existed before the update?
What parameter value exists after the update?
Was the old gradient cleared before the next backward pass?
Is this still the same parameter object?

高价值测量

before = p.detach().clone()

loss.backward()

grad = None if p.grad is None else p.grad.detach().clone()

optimizer.step()

delta = p.detach() - before

受控朴素 SGD 步(无动量、无 weight decay)下:

delta ≈ -learning_rate * gradient

deliberate 受限实验下等式不成立,更新机制不是你以为的那个。

成立,别盲目 generalize 到 Adam、动量、weight decay 等有 state 更新规则。

停于

分别证明:

the forward calculation is the intended one
backward produced the expected derivative
the intended parameter object moved
old gradients are not contaminating the next step

再往系统的下一层。


2. 张量:找第一个错的张量,不是第一个不合法的

张量不只有值。

有用的描述是:

shape
dtype
device
stride
contiguity
axis meaning
value meaning

前五张量上可查。

后两属于程序的契约。

第一步

算子跑前写下期待的张量。

不是:

I expect something like [32, ?]

而是:

[B, T, D]
B = examples
T = positions
D = feature coordinates

再每边界期待对观测比。

布局问题用确定性值

随机数藏重排。

reshape、转置、头拆分、置换 bug,用:

x = torch.arange(...)

小到能打印。

两算子产出同样形状,看值去了哪。

常见 false 修复

reshape until the exception disappears
change in_features to match the tensor that arrived
change in_channels to match an NHWC batch
squeeze dimensions without naming what they represent

这些能把上游错误变成合法的下游结构。

停于

找到观测张量事实或轴语义第一次不再对上期待契约的第一边界。

在那里修。


3. Autograd:找依赖通路停止存在处

参数没收到期待梯度,别从终 .grad 开始。

从问题开始:

跑出的图里这个损失真依赖这个参数吗?

autograd 记录这次前向跑过的算子。它不记录你想跑的模型。

证据梯子

parameter.requires_grad
parameter.is_leaf
loss.requires_grad
loss.grad_fn
parameter.grad after a controlled backward pass
reachability from loss back to expected leaves

健康的 loss.grad_fn 证明损失有可微历史。

它不证明每意图参数都在那历史上。

区分

grad is None

与:

grad is a tensor of zeros

.grad = None 开始的受控 pass 下:

  • None 意思是该参数没累积梯度;
  • 零张量意思是 autograd 物化了累积导数为零的梯度。

不同机制。

通路缺席的常见病因

detach()
torch.no_grad() around part of the forward computation
a branch that did not execute
a parameter that was not used
rebinding to a new non-tracked tensor

停于

指出意图依赖不再存在的第一算子或分支。


4. Raw 神经网络:六断言里哪个挂了

第 4 章确立“梯度看起来 fine”只回答一个问题。

机械跑着的网,六断言分类失败:

Claim Question Useful evidence
Representation 结构表达得了要的函数吗? 代数、证伪探针、更简单基线
Computation 前向算的是意图结构吗? 中间张量、形状/值对比、独立实现
Objective 损失编码了意图问题吗? 手工损失检查、可信参照、logits/目标契约
Differentiation 每意图参数到损失有通路吗? autograd 可达、.grad
Update 意图参数真变了吗? 参数快照与 delta
Evaluation 度量的是真在乎的行为吗? 留出数据、有意义基线

为什么重要

这些失败看起来像:

missing nonlinearity
symmetric initialization
zero initialization
detached parameter
healthy gradient on an un-updated parameter
wrong objective

单梯度统计分不开所有。

停于

说得出证据支撑哪个断言、哪个断言仍无支撑。

别拿一断言的证据当另一断言的证明。


5. nn.Module:比四套结构

东西好似消失,停止把“模型”当一东西想。

有四重叠结构:

1. Python object graph
2. registered module tree
3. autograd graph
4. optimizer parameter groups

回答不同问题。

Operation Structure it consults
forward() through normal Python attribute access Python objects
named_modules() registered module tree
named_parameters() registered parameters
named_buffers() registered buffers
state_dict() registered parameters + persistent buffers
.to(device) registered parameters + buffers
train() / eval() registered modules
loss.backward() autograd graph created by execution
optimizer.step() that optimizer’s parameter groups

诊断问题

出故障的操作遍历哪套结构,我的对象真在里面吗?

张量能:

require gradients
receive gradients
participate in forward

还不:

registered
saved
moved by model.to(...)
owned by the optimizer

对象同一性重要

优化器构造后 head 被换,优化器可能仍拥有旧参数对象。

名字形状看着对。

同一性错了。

停于

reconcile:

what Python can reach
what PyTorch registered
what autograd reached
what the optimizer owns

失败相关的每对象。


6. DataLoader 与输入供给:生产者消费者拆开

“GPU 利用率低”不是 DataLoader 诊断。

先问训练循环是不是在等 batch。

最小有用测量是:

t0 = perf_counter()
batch = next(iterator)
t1 = perf_counter()

train_step(batch)
t2 = perf_counter()

现在有两不同观测:

wait = t1 - t0
work = t2 - t1

建模流水线

producers
    ↓
queue / buffering
    ↓
consumer

再分开:

producer throughput
consumer throughput
buffering
end-to-end throughput

更多 worker 重叠等待。

变不出已饱和的 CPU 资源。

尾也要记,不只平均

batch 等待,捕获:

median
p95
maximum

mean 低尾大,消费者照样反复饿。

一次 sweep 一 control

比如:

num_workers = 0, 1, 2, 4, 8

记:

examples/s
batch-wait distribution
post-batch work time

等待降而消费者 work 涨、端到端吞吐不变,worker 在和消费者争同样资源。

停于

说得出实际 limiting 的资源或阶段:

read
decode
transform
collate
process startup
CPU contention
host-to-device transfer
consumer compute

再修那个机制。


7. 变换与预处理:跟踪一样本

预处理流水线能产出合法而含义错的张量。

第 7 章把输入契约分成五类:

1. STRUCTURE
2. NUMERIC INTERPRETATION
3. SEMANTICS
4. RELATIONSHIPS
5. PROVENANCE

要不同仪器。

Category Typical evidence
Structure type, rank, shape
Numeric interpretation dtype, range, scale, normalization state
Semantics known-sample probe, direct inspection
Relationships image-mask/box alignment, input-target assertion
Provenance recorded split, fitted statistics, checkpoint/preprocessing version

方法

找到样本第一次不再满足意图输入契约的变换边界。

从:

one sample
one process
no batching
controlled randomness

开始逐阶段走。

重要局限

全绿 metadata 报告不是健康证明。

张量说不出:

its RGB channels were interpreted as BGR
the augmentation invalidated the label
the mask no longer aligns with the image
normalization statistics were fitted on validation data
the serving tokenizer differs from training

那些要显式探针或记录。

停于

认出五契约类里最早失败的阶段。

到那时再引入 batch、worker、性能复杂度。


8. CNN 几何:跑前推导

卷积网两问题分开:

CHANNEL QUESTION
C_in → C_out

SPATIAL QUESTION
H_in,W_in → H_out,W_out

C_out 声明的。

空间尺寸推导的。

方法

层跑之前推导期待形状。与观测形状比。调查第一次不一致。

用台账:

stage
operation
observed input
expected output
observed output
first divergence

终 Linear 永远别先修

对:

mat1 and mat2 shapes cannot be multiplied

倒推:

Where did the flattened feature count come from?
Which earlier layer changed C, H or W?
Was that change intended?

上游几何变化是故意的,改 in_features 才对。

记住语义极限

形状台账证明组合。

它不证明高宽没互换、预处理保住含义、结构适合任务。

停于

每改形状的算子处推导与观测几何一致,直到失败边界。


9. 特征空间:解释数前先说出被表示对象

张量形状:

[B, T, D]

问:

What does one D-vector represent?
How many such vectors are present?
Which axis contains its coordinates?

nn.Linear(D_in, D_out) 变换末轴并保留前导轴。

所以 [B,T,D] → [B,T,1] 意思是每(样本,位置)一分,不是一样本一分。

解释标量前先命名

dot product
cosine similarity
Euclidean distance
classifier score
signed distance to a hyperplane

不可互换。

检查不变式

问什么在下面不变:

one vector is rescaled
w and b are scaled together
one feature coordinate changes units

不该影响解释的东西动了量,多半量的名字错了。

检查器

表示的有用行包括:

vector norms
per-coordinate scales
non-finite values
pairwise cosine
collapse toward zero
collapse toward one direction
dominant coordinates

停于

说得出:

what one vector represents
which axis holds its coordinates
what operation acts on those coordinates
what numerical quantity came out
which invariances that quantity has

10. 注意力:四层正确性

注意力要的不只形状检查。

用四层:

LEVEL 1  SHAPE
         Does the tensor have the dimensions I derived?

LEVEL 2  AXIS SEMANTICS
         Do those dimensions contain the objects I intended?

LEVEL 3  NUMERICAL INVARIANTS
         Do valid query rows normalize over keys?
         Are forbidden weights zero?
         Are values finite?

LEVEL 4  BEHAVIOR
         Does a future-token intervention leave the prefix unchanged?
         Does the implementation agree with a trusted reference?

方法

derive the axes
    ↓
execute one stage
    ↓
verify the invariant introduced by that stage
    ↓
stop at the first failure

高价值检查

merge(split(x)) == x
scores shape == [B, Nh, Tq, Tk]
softmax normalizes over Tk
forbidden weights == 0
no fully blocked query unless explicitly supported
merged layout restores [B, Tq, E]
future-token intervention preserves the causal prefix within tolerance

Level 4 为什么存在

缺 1/sqrt(Dh) 缩放能保留:

shape
axis semantics
finite values
row sums

还与意图注意力算子实质不一致。

停于

知道挂的最早正确性层与阶段。


11. 跑而不学的训练:走学习链

学习链是:

TASK
  ↓
OBJECTIVE
  ↓
DEPENDENCY
  ↓
GRADIENT
  ↓
OPTIMIZER OWNERSHIP
  ↓
PARAMETER UPDATE
  ↓
CAPABILITY

这个工具适合:

forward runs
loss is finite
backward runs
gradients exist
optimizer.step() runs

and the model is still useless

TASK

Does the target still describe the input?

OBJECTIVE

Are the outputs in the representation the loss expects?
Can one example reproduce the loss manually?

DEPENDENCY

Does the loss depend on the intended parameters?

GRADIENT

Are expected gradients present?
None, zero, finite, non-finite?

OPTIMIZER OWNERSHIP

Does the optimizer own those exact parameter objects?
Does it own stale objects that are no longer in the model?

PARAMETER UPDATE

Did the intended objects actually move?

CAPABILITY

Can the controlled system overfit a tiny fixed batch?

重要警告

调 backward() 与 optimizer.step() 的学习台账是改 state 的探针。

拷贝/检查点上跑,或 deliberate 接受牺牲步。

停于

找到第一个没出现的期待后果。

在那里调。

早 link 无支撑时别调学习率、结构、精度、正则。


12. 性能:优化前先定义比较

性能不是一行代码的属性。

它是契约下的结果:

workload
hardware
software/runtime
execution mode
metric
measurement boundary

先选问题

latency
throughput
peak memory
cold-start cost
steady-state cost

这些能反向走。

方法

Measure → localize → intervene → remeasure.

性能台账

捕获:

WORKLOAD
  model / operation
  input shape
  dtype
  device
  mode
  eager / compiled

MEASUREMENT
  warmup
  repeats
  synchronization
  metric

TIME
  input wait
  transfer
  forward
  loss
  backward
  optimizer
  logging / synchronization

MEMORY
  allocated
  reserved
  peak allocated
  peak reserved

CUDA 计时

设备异步时,无完成边界的主机 wall-clock 计时测的可能是提交而不是完成的设备 work。

知道在测:

host-visible latency
device elapsed time
end-to-end workload time

内存

分开:

allocated     live PyTorch-managed allocations
reserved      allocator-managed pool
peak          transient high-water mark

memory_allocated() 在相同迭代 climb,活 PyTorch 分配集在长大。跟踪归属。

单凭保留内存高别叫活张量泄漏。

Profiler 规则

profiler 管定位。

无 profile benchmark 管最终性能比较。

停于

说得出:

the metric that matters
the phase/resource that dominates it
the mechanism hypothesis
the one intervention that tests that hypothesis
the correctness property that must remain unchanged
the uninstrumented before/after result

13. torch.compile:检查编译器的 hidden state

返回正确数的编译调用证明了正确。

它没证明:

the hot region was captured
the graph was reused
guards remained valid
recompilation stopped
steady-state performance improved
cold cost amortized

有用的 progression 是:

RUNS
  ↓
CORRECT
  ↓
CAPTURED
  ↓
STABLE
  ↓
WORTH IT

从 eager 开始

eager 已错,编译不是第一问题。

再比正确性

用 workload 恰当的陈述 tolerance。

再看捕获

有用证据包括:

graph-break logs
fullgraph=True as a capture test for the exact call
structured compiler summaries

版本里没有一仪器自动 ground truth。工具分歧时三角验证。

再看稳定

捕获重编译证据。

问:

Which guard failed?
shape?
dtype?
device?
Python value?
Did the workload legitimately vary on that dimension?
Did a cached specialization get reused?
Did a new specialization get created?
Did the cache limit force an unmatched call to eager?

再问值不值

分开:

cold first call
warm-cache first call
steady-state call
recurring recompilation

启动代价重要时算摊销。

停于

说得出不再成立的编译器假设,或确立编译稳定、剩下是普通 workload 经济学。


14. 回归:解释前先确立差

新指标差了。

那是观测。

还不是回归机制。

前三可能

NOTHING MEANINGFUL CHANGED
ordinary run-to-run variation

MEASUREMENT CHANGED
different evaluation path / split / preprocessing / mode

SYSTEM CHANGED
code, configuration, data, environment, compiler state

解释第三前排除前两。

基线 variation

没改配置多种子跑。

报告分布,不只一数。

观测 min/max spread 是 context。

它不是通用统计检测阈值。

配对 valid 时配对

共享种子在两跑保留足够共享随机让种子 effect 抵消时,subtle 比较更敏感。

看:

delta(seed_i) = B(seed_i) - A(seed_i)

不只比两独立范围。

验证评估契约

记到回答:

same validation examples?
same preprocessing?
same aggregation?
same model.eval() state?
same checkpoint-selection rule?

fingerprint 只证明 hash 进去的字段。

记录 run

至少:

code version
config
data identity
seed
environment
evaluation contract
compiler state if relevant
result

小心二分

二分 assumed 好/坏 predicate 行为合适。

变化能交互。

某变化被边界 implicates 后,候选单独施加确认 effect。

停于

重复证据确立值得解释的差,测量契约已知可比。

到那时再调查机制。


15. 完整模型:路由,不发明新仪器

capstone 的课是单诊断 certify 不了大模型。

完整系统同时含前面每系统:

text
  ↓
tokenization
  ↓
batch/target alignment
  ↓
embeddings
  ↓
attention
  ↓
residual blocks
  ↓
logits
  ↓
objective
  ↓
autograd
  ↓
optimizer
  ↓
performance / compilation
  ↓
generation
  ↓
evaluation and comparison

全模型挂,路由症状。

例子:

Generated text is nonsense but training loss is excellent
    → TASK / target alignment first

A future token changes an earlier logit
    → attention Level 4 intervention

Initial loss is wildly inconsistent with the declared initialization
    → OBJECTIVE / initialization contract

Gradients are finite but the head never moves
    → optimizer ownership + parameter delta

Training is correct but throughput is low
    → performance ledger

Compiled model pauses on new sequence lengths
    → guards / recompilation

New configuration improves one run by 0.001
    → baseline variation + paired comparison

大模型不让前面仪器 obsolete。

它让选对仪器更重要。


症状到证据野战表

这张表故意与逐章指南冗余。反方向用:从观测出发。

Symptom Tempting explanation Evidence to collect first
mat1 and mat2 shapes cannot be multiplied “Linear 层的 in_features 错了。” 推导每上游形状,找到第一次几何分歧。
expected ... channels, got ... “Conv2d.in_channels 错了。” 说出进轴含义。改层前检查 NHWC vs NCHW。
.grad is None “忘了 requires_grad。” grad=None 开始,跑一次反向,再看图可达。
.grad 有限但参数不动 “学习率太小。” 查优化器对象同一性与测得的参数 delta。
损失降但任务行为错 “模型容量不够。” 已知样本验证输入目标对齐与目标。
小 batch 过拟合过了 “整个训练系统对了。” 记住它没证明的:任务语义、验证契约、泛化。
GPU 利用率低 “加 num_workers。” 输入等待、传输、前向、反向、优化器时间分开测。
nvidia-smi 内存高 “内存泄漏了。” 固定形状重复迭代比已分配、保留、峰值。
empty_cache() 让可见内存降了 “泄漏修好了。” 活已分配内存变没变。
注意力行形状对 “注意力对了。” 查头往返、key 轴归一化、forbidden 权重、干预行为。
torch.compile 返回正确输出 “编译 work 了。” 捕获、守卫、复用/重编译、workload 经济学分开测。
编译代码更慢 “torch.compile 对这个模型不好。” 冷代价、热代价、recurring 重编译、稳态分开。
commit 间验证变了 “代码改动导致的。” 先确立基线 variation,验证评估契约。
同种子跑出不同 “PyTorch 无视种子。” 比线程/设备/版本/RNG 来源与确定性配置。
两跑终指标相同 “等价。” 比轨迹与预先声明的检查点/评估政策。

一 bug 规则

几失败同时,手修一边界重跑。

别一次改:

input layout
layer dimensions
learning rate
optimizer
normalization

在一次 edit。

多改修复毁掉因果信息。

本书反复用的序列:

find first divergence
fix one thing
rerun
allow the next divergence to reveal itself

每 edit 慢,诊断快。


观测与干预是不同算子

仪器能改变系统。

例子:

hooks can alter lifetime and add overhead
profiler instrumentation changes timing
.item() can synchronize CUDA
printing values inside compiled code can create graph breaks
a diagnostic optimizer step mutates optimizer state
switching train/eval changes module behavior
seeding changes stochastic execution
gradient clipping changes the gradient you were trying to inspect

叫观测前问:

这个仪器仅 present 就改了什么?

答案重要,终验证去掉仪器再跑。


已知好参照 disproportionately 值钱

难推理的算子,把参数 reduce。

用:

manual arithmetic versus PyTorch
manual loss versus F.cross_entropy
manual attention versus scaled_dot_product_attention
eager versus compiled
healthy preprocessing versus candidate preprocessing
healthy run versus candidate run at shared seeds

参照对比在每没控变量 hold 固定时最强。

输出不一致,按顺序比中间阶段。

第一次不一致通常比终误差大小有用。


区分两解释的最小实验

好的调试实验不只让一假设看起来 plausible。

它让两解释预测不同结局。

例子:

缺失梯度

假设:

A. graph path is broken
B. path exists but local derivative is zero

实验:

clear grad to None
run one backward pass
inspect whether grad remains None or becomes a zero tensor

GPU 利用率低

假设:

A. input pipeline is starving the device
B. model/host execution is the bottleneck

实验:

measure batch wait separately from post-batch work

编译慢

假设:

A. steady-state kernel is slower
B. recurring recompilation dominates

实验:

measure a repeated fixed-shape steady state
capture recompile logs on the real varying workload

Apparent 回归

假设:

A. model behavior changed
B. ordinary seed variation explains the observation

实验:

run both versions on shared seeds and inspect paired deltas

最好的实验通常不是仪器最多的。

是结局迫使假设分开的那个。


可复用事故工作单

抄进 issue、notebook、调试 session。

PYTORCH INCIDENT

SYMPTOM
What exactly was observed?

EXPECTED
What specific value, shape, behavior or metric was expected?

REPRODUCTION
What is the smallest input / batch / configuration that reproduces it?

FIRST CONTRACT
Which contract should be checked first?
    tensor
    graph
    ownership
    input representation
    geometry
    attention
    learning chain
    performance
    compiler
    run comparison

OBSERVATIONS
What did the instrument actually report?

FIRST DIVERGENCE
Where does observed behavior first disagree with the expectation?

HYPOTHESES
What are the smallest two or three mechanisms consistent with that evidence?

DISTINGUISHING EXPERIMENT
What one experiment makes those hypotheses predict different outcomes?

INTERVENTION
Change exactly one mechanism.

MUST CHANGE
Which measurement should change if the diagnosis is correct?

MUST NOT CHANGE
Which correctness property must remain unchanged?

VERIFICATION
Did the original failure disappear for the predicted reason?

REGRESSION TEST
What small test should remain so this exact failure cannot silently return?

MUST CHANGE 与 MUST NOT CHANGE 填不出,proposed 修复大概还太 vague。


AI 当诊断伙伴

本书最有用的通用 prompt 不是“修 PyTorch 模型”。

用接近这个的:

Here is a failing PyTorch system.

Do not rewrite it and do not propose fixes yet.

1. State the expected contract at each relevant boundary.
2. Tell me which facts I can inspect directly and which require a semantic,
   behavioral or provenance probe.
3. Identify the smallest instrument from this list that can localize the failure:
      tensor trace
      autograd reachability
      six-claim audit
      four-structure audit
      producer-consumer timing
      preprocessing stage trace
      CNN shape ledger
      feature-space inspector
      attention four-level check
      learning ledger
      performance ledger
      compiler guard/recompile evidence
      paired run comparison
4. Ask for the missing evidence.
5. Identify the FIRST boundary whose evidence contradicts the intended system.
6. Give at most three mechanisms consistent with that boundary.
7. Propose one experiment that distinguishes those mechanisms.
8. State what should change if the diagnosis is correct.
9. State what must remain unchanged.

Do not propose a repair until the first divergence is supported by evidence.

prompt 的目的不是让助手谨慎。

是让助手可证伪。


何时升级

从便宜开始。

便宜仪器过了再升级。

print / assert tensor facts
        ↓
known-sample contract
        ↓
graph / ownership inspection
        ↓
controlled one-step probe
        ↓
tiny-batch capability test
        ↓
hooks / detailed activation inspection
        ↓
profiler / allocator snapshot / compiler logs
        ↓
multi-seed comparison

这不是通用代价排序。小 workload 种子 sweep 能便宜,一步 profiler 能便宜。

原则是:

用区分当前假设的 least invasive 仪器。


何时停调

四句话全 true 时停:

1. You can name the mechanism that produced the symptom.

2. The repair changed the measurement the mechanism predicted.

3. An independent correctness property remained unchanged.

4. The failure can now be expressed as a test, assertion, contract or recorded
   comparison that would catch it if it returned.

“好似 work 了”的模型没达标。

诊断达标了。


完整诊断栈

本书从一标量参数开始,到装不进脑子的模型结束。仪器按同样顺序累积。

TRAINING LOOP
    forward / backward / update

TENSOR
    first wrong tensor

AUTOGRAD
    first broken dependency edge

RAW NETWORK
    representation / computation / objective /
    differentiation / update / evaluation

MODULE STATE
    Python graph / registered tree /
    autograd graph / optimizer groups

INPUT DELIVERY
    producer / queue / consumer

PREPROCESSING
    structure / numeric interpretation /
    semantics / relationships / provenance

CNN GEOMETRY
    derive / trace / first shape divergence

FEATURE SPACE
    represented object / feature axis /
    quantity / invariance

ATTENTION
    shape / axis semantics /
    numerical invariants / behavior

LEARNING
    TASK → OBJECTIVE → DEPENDENCY → GRADIENT →
    OPTIMIZER OWNERSHIP → PARAMETER UPDATE → CAPABILITY

PERFORMANCE
    measurement contract → phase/resource →
    intervention → uninstrumented remeasurement

COMPILATION
    RUNS → CORRECT → CAPTURED → STABLE → WORTH IT

REGRESSION
    establish the difference →
    verify the measurement →
    localize what changed

CAPSTONE
    choose the instrument that owns the symptom

底没有终极通用检查器。

这就是要点。

不同失败违反不同契约,危险的常常保住下面每更简单的契约。

张量能形状对含义错。

注意力块能外部维度全保留而比错对象。

参数能梯度健康从不动。

模型能损失最小化学错任务。

benchmark 能为错测量返回精确数。

编译模型能数值正确、灾难性慢。

新跑能变差而有意义的东西什么都没回归。

common 技能不是背修复。

是知道当前症状还没给出什么证据。

让隐藏结构可见。找到第一个分歧。修复造成它的机制。验证系统真变好了。