Read the Product, Not the Code

The introduction to this book describes a walk. A research paper, an idea for a trading system, and in about three minutes a couple of thousand lines of code from GPT-4. I glanced at it, recognised that a substantial amount of work had been done, and was fairly sure it would run. Then I went home and ran it. With a few minor changes, it worked.

Notice what I did not do. I did not read two thousand lines before deciding whether the work was any good. I did not review it the way I had reviewed code for most of my career, line by line, looking for the mistake. I found out whether it worked by using it.

And notice what that did not tell me. It did not tell me whether the trading strategy was any good. Running the program established that the program ran. It said nothing about whether the idea inside it would make money, and no amount of playing with it would have.

Both halves of that story are the subject of this chapter. The first half is a change in where the human looks: at the product, not at the code. The second half is why that change does not abandon engineering: the properties you cannot see by using a product still have to be established, but increasingly by machines, not by you reading source files.

Stop looking at the code

For most of the history of professional software, the code was where quality lived. You judged a programmer by their code. You judged a change by reading its diff. The pull request, the code review, the second pair of eyes on every line: these were the central quality rituals of the profession, and for good reason. When humans wrote the code, humans reading it was the best defence we had.

Machine coding moves that defence. Not because reading code has stopped being useful — it has not — but because a human reading every line is no longer the cheapest, the fastest or even the most reliable way to find what is wrong with machine-written software. Machines can read code too. They can read all of it, every time, without fatigue, and they can run what they read. The machine that writes the code can be checked by other machines that read it.

That leaves the human free to look somewhere else. I think the right place to look is the thing being created.

I want to be precise about the claim. I am not banning code review, and I am not saying nobody will ever need to read source again. I am saying that, for a growing share of software work, reading the code should stop being the developer’s primary way of judging whether the work is good. The primary question changes from “has the machine written good code?” to “have I created the thing I wanted?” — and the first question is increasingly one the machinery can answer on my behalf.

Code that is not yours

Chapter 12 described the linker files and parser generators that once demanded my attention and then, for most of my work, quietly stopped. The machinery still runs. I no longer read its output. I suggested there that much ordinary source code may follow the same path.

One consequence deserves attention here, because it unsettles experienced programmers more than any other. As machines write more of the implementation, the code will increasingly not look like yours. It will use structures you would not have chosen, conventions you do not recognise and an organisation that made sense to the machine and not necessarily to you. Open it and you may feel like a guest in your own project.

That is not, by itself, a failure. The test of an implementation was never whether it matched one person’s reading habits. It was whether it worked, whether it could be changed safely, whether it could be verified and whether it would survive the people and machines that maintain it next. Those requirements do not go away. Maintainability still matters — arguably more, since the machine maintaining the code next month needs a coherent structure to work in. What changes is who the code is written for. It no longer has to be written for you to read. It has to be written so that it can be verified, changed and kept working, by whoever and whatever does that work.

I do not claim that unreadable code is a virtue, or that generated code is already beyond human understanding. Much of it is perfectly readable. My point is narrower: the fact that I personally would have written it differently is no longer evidence that anything is wrong.

Two readers

Here is the picture I now work with. There are two readers of every machine-built application, and they read different things.

The machine reads the code. It writes the implementation, generates and runs the tests, reads the diffs, inspects its own changes and other machines’ changes, analyses failures and reports what it found. This is the layer where correctness of the parts is checked: does this function do what it claims, did this change break something that used to work, is there an obvious security mistake.

I read the product. I use it, look at it, and judge it against what I meant. This is the layer where fitness of the whole is judged: does this do what I wanted, does it behave the way I expected, does it feel right, is anything missing, is anything there that should not be.

Neither reader can do the other’s job well. I cannot reliably find a subtle off-by-one error by playing a game. The machine cannot reliably tell me that the game has become dull. The mistake in machine coding is to collapse the two — to treat the machine’s report about the code as a verdict on the product, or to treat my enjoyment of the product as a verdict on the code.

The machine’s layer is already becoming tooling. Roborev, an open-source tool I have used, is a good example of where this is heading: it runs locally, reviews every commit automatically through a Git hook using the coding agents you already have, records its findings, and can hand those findings back to an agent to fix and then review the fix again, looping until the reviews pass. I mention it as an example of a category, not as a requirement or an endorsement of an infallible reviewer. Two machines agreeing can still share the same blind spot, and Chapters 17 and 18 take reviewer independence seriously. But the direction is plain. Code review — the profession’s central quality ritual — is moving inside the machinery. It is becoming something that happens on every commit, continuously, without a human performing it.

That is, in the end, what the title of this book means. The machine does the coding. Increasingly, the machine also reads the code. What is left for me is the product.

Use, observe, describe, change

The practice at the product layer is a loop simple enough to state in a line: use it, observe it, describe what you observed, let the machine change it, and use it again. It is the book’s master loop — communicate, build, evaluate, improve — seen from the side of the person holding the product.

Use means the real thing, not the machine’s account of the real thing. A summary that says “the game now feels faster” is a claim. Playing it is the evidence.

Observe means paying attention to what actually happens, including the small things: where you hesitated, what you looked for and could not find, what you stopped noticing, what made you smile. Observation covers behaviour, appearance, responsiveness, clarity and the harder-to-name sense of whether the thing is pleasant to use.

Describe is where most people go wrong, so it deserves more space than the others.

The temptation, especially for programmers, is to describe the fix rather than the observation. “Increase the enemy spawn rate to one every eight hundred milliseconds” is an implementation directive dressed as feedback. It assumes you already know what is wrong and how to correct it, and it denies the machine the chance to find a better correction. Compare: “The first thirty seconds are boring — nothing much happens until I’ve already lost interest.” That is an observation. It says what you experienced and what it cost you. The machine can then decide whether the answer is more enemies, faster enemies, a different opening, or something you would not have thought of.

Good descriptions share a few qualities. They say what you saw, not what you suspect in the source. They say what you wanted instead, or at least why what you saw was wrong. They use comparisons freely — “it feels like a screensaver”, “I want it to feel like the last minute of a close match” — because a good comparison carries more intention than a paragraph of adjectives. And they are specific about where: which screen, which moment, which step of the journey.

There are times to be prescriptive. If you know exactly what you want — this button here, that colour there — say so. The point is not to forbid precise instructions. It is to stop reaching for implementation language when what you actually have is an experience.

Change is the machine’s part, within whatever boundaries you have set. It may alter code, tests, styles, configuration, assets. You do not need to follow how. Then you use it again, and ask the only questions that matter at this layer: is it better, is anything newly broken, and is it closer to what I meant?

Shaping behaviour and feel

Most of the work of making something good happens in this loop, in many small turns rather than a few large ones.

Some turns are about behaviour. A website has to support the journeys people actually take, not the ones its designer imagined. A workflow has to produce the right outcome from realistic inputs, not tidy examples. A research tool has to return answers that are relevant and grounded, and you find out whether it does by asking it real questions. Using the product is also where new requirements appear. You do not discover that the scheduling tool needs to handle cancellations until you try to cancel something.

Some turns are about appearance and feel. You may not know anything about layout systems, animation or colour theory, and you do not need to. “Make this quieter.” “The important number should stand out.” “There are too many steps between opening this and doing anything.” “The movement feels sluggish.” Each is enough for the machine to act on, and a sequence of such turns, each observed, each described, can produce something with a character of its own. Chapter 16 asks what makes that character genuinely good. Here the point is only that you can shape it without touching the machinery.

And some of the best turns remove things. Using the product is how you discover that the settings screen nobody opens, the second navigation bar, the explanatory paragraph nobody reads, are making it worse. Chapter 11 dealt with what happens when additions accumulate unchecked. In ordinary practice the remedy is simpler: when you see something that does not earn its place, say so.

Reading evidence you did not produce

If the machine reads the code, you will increasingly receive reports about code you have not read: test results, review findings, analysis summaries, claims that something was fixed. This is a new skill for most people, programmers included, and it is the practical heart of trusting the machine’s layer without returning to it.

You do not need to understand the code to read a report about it intelligently. You need to ask what the evidence actually covers. Four questions do most of the work.

What was checked? “All tests pass” means the tests that exist pass. It says nothing about behaviour the tests do not exercise. Ask what the tests cover, in terms of what the product does, not in terms of files.

Who checked it? A review by the same agent that wrote the change is worth less than a review by a separate one with no stake in the result. Neither is worth as much as a check that actually ran the product.

Against what? A review against the code alone finds code problems. A review against your stated intention can find the gap between what was built and what was meant. Ask which it was.

What was not checked? This is the question machines answer least readily and the one that matters most. A trustworthy report says what it could not establish. If a report has no limitations section, ask for one.

When a finding matters and the report is ambiguous, ask the machine to demonstrate rather than assert: show me the failing case, then show me it passing. You are still not reading code. You are reading evidence, which is a different and more transferable skill.

What the product cannot tell you

Using a product reveals a great deal. It does not reveal everything, and the limits are worth stating plainly.

A game can feel excellent and contain a security hole. A website can look right and mishandle private information. A trading system can run flawlessly and embody a strategy that loses money — my walk established that the program worked, not that the idea did. A service can behave perfectly for one user and collapse under a thousand. A calculation can be right for every example you tried and wrong for the one you did not.

None of these limits sends you back to reading every source file. They mean the product needs evidence of properties you cannot see by using it: tests aimed at those properties, security checks, load tests, independent review, and, for questions like the trading strategy, evidence from the domain itself rather than from the software. Chapter 14 separated your experience of the product, technical verification, a machine’s critique and independent evidence. Add one more line to that list: the decision to release or deploy. That decision, where the consequences are real, rests on all of the above, and it remains a decision someone has to own.

Most of the evidence can be produced by machines. What you cannot delegate is understanding what the evidence means and where it stops — which is exactly why reading evidence is a skill worth learning.

Try Machine Coding — Tune the Game, Not the Code

Use the game from Chapter 13. If you have not built it yet, this is a proposed exercise rather than one you can complete today; either way, nothing below reports results from a session of mine. Speak each instruction and carry the conversation on naturally. One rule throughout: do not open the code.

Play it.

Open the current version of my game. I want to evaluate it as a player. Tell me how to play the latest version and confirm that it’s the one you last changed.

Play several rounds before saying anything else.

Describe what you noticed — not what to fix.

Here’s what I noticed as a player. [Say it in your own words: what dragged, what confused you, what you liked, where you hesitated.] Before changing anything, tell me what you think is causing each of those, and suggest more than one way to address the ones that matter.

Compare your description with the machine’s diagnosis. Did it understand what you experienced?

Direct the change, then play again.

Make the changes we agreed. Keep the existing controls and scoring behaviour. Run the relevant checks and tell me when there’s a new version to play.

Play it. Is it better? Is anything newly wrong? Repeat this step as many times as it takes.

Let the machinery read the code.

Now have a separate reviewer — not the agent that made the changes — inspect the implementation and its tests for defects and regressions. Report which findings were confirmed, which were fixed, which remain open, what was checked, and what was not checked.

Read the evidence, then decide.

Walk me through that report in terms of what the game does, not in terms of code. For anything that was fixed, show me how I’d notice the difference as a player.

Then play the game one last time and decide whether to accept this version. Your decision should rest on two things: what you experienced, and what the evidence covered. Neither alone is enough.

Your job is to recognise the result

I want to create applications. I do not particularly want to write their code, read all of it, or maintain their tests, documentation and build scripts myself. I want the machine to do as much of that as it can reliably do, and to tell me honestly where its evidence stops.

What I want to do is experience what has been made. See how it behaves. Find what is missing. Change what I do not like and improve what I do. Make it work the way I meant it to work. And when something matters that I cannot see by using it, I want the machinery to establish it and show me the evidence in terms I can judge.

That is not abandoning engineering. It is changing where my attention goes. The code is increasingly the machine’s responsibility, in both directions: it writes it, and it reads it. The creation is mine.

Which raises the question the next chapter has to answer. If I am judging the product rather than the code, what am I judging it against? Working is not the same as good, and good is not the same as distinctive. Chapter 16, A New Approach to Quality, is about the standard behind the judgement.