Debugging AI

Debugging AI builds one continuous debugging system from Python tracebacks to production AI incidents.

The progression is deliberate:

1. Debug values and state in deterministic software.
2. Cross the bridge: debug the interactive and numerical systems in between — notebooks, data, tensors, training, and evaluation — where state accumulates invisibly and every result must be reproduced before it is trusted.
3. Debug distributions and uncertainty in learned systems.
4. Debug intent and evidence in prompts, retrieval, and hallucinations.
5. Debug trajectories in agents.
6. Debug causes experimentally and convert diagnoses into prevention.

The debugging objects underneath that progression stay fixed — values, state, distributions, evidence, trajectories. Step 2 is not a new object; it is state stretched from a single binding to an entire accumulated computation, and the on-ramp to the distributional thinking step 3 depends on.

By the end, the reader should be able to move from a symptom to a scoped, reversible, causally-supported diagnosis, design discriminating experiments, and convert confirmed failures into durable regression artifacts.

The five objects are deep-structure categories, not a taxonomy of convenience: skilled diagnosis increasingly depends on classifying the structure of the failure — a value, an accumulated state, a distribution, an evidence chain, a trajectory — rather than reacting to its surface symptom, and each object selects its diagnostic method.

A diagnostic root-cause account, in this book, is an evidence-backed, reversible explanation sufficient to predict the observed failure within a defined diagnostic envelope. It is built by climbing a ladder — a difference, then a relevant difference, then a difference that changes the outcome when changed, then one that survives replication and reversal. It is not a claim that every system exposes one objective earliest cause: where ordering, observability, or intervention break down, the honest output is PROVISIONAL or INCONCLUSIVE, and socio-technical incidents carry a separate list of contributing factors rather than a single root.