The Measurement Instrument

Mon, 14 Sep 2026 05:00:02 +0100

Chapter 1 defined memory behaviourally: the past must change what the system does now, ideally improving it. This chapter defines how that claim is measured. The distinction matters from the first paragraph, because the word “instrument” will recur throughout the book and it never means the memory architecture. The memory system is one product of this book. The Memory Measurement Instrument — the apparatus used to determine whether that system remembers — is the other.

Change One Thing

Fri, 28 Aug 2026 11:05:00 +0100

The previous three chapters ran three optimizers against a metric we had already proved was blind, and got what chapter 9 predicted: a search that pursued the objective faithfully and damaged the thing the objective could not see.

That result has an obvious interpretation and a correct one, and they are not the same.

The obvious interpretation is that the optimizers underperformed. The correct one requires an experiment, because “the objective was the problem” is a hypothesis and we had not tested it. Everything we knew was consistent with a second explanation — that DSPy’s optimizers do not do much on a small local model regardless of what you point them at.

Experimental Design on Programmer.ie: Modern AI programming

The Measurement Instrument

Change One Thing