Part 1 — Where You Stand

The missing operation

CodeAI has a call whose shape is almost disappointingly ordinary:

result = adapter.invoke(spec)

It sends one prepared request through the OpenCode gateway and returns a CallResult. It does not open a conversation window. It does not know which chapter you are writing. It does not decide whether the returned paragraph belongs in the manuscript.

That small separation is where this book begins.

Here is the work it separates. You are reviewing a paragraph for unsupported factual claims. You copy the paragraph into a chat window, explain what counts as support, and ask for a review. The model flags a sentence that needs a source. You find the source, decide whether it actually supports the sentence, and either add a citation or rewrite the sentence.

That is a useful process. It is also much larger than the two messages visible in the window.

You chose the paragraph and remembered which manuscript it came from. You supplied the review criteria, distinguishing a suggestion from a correction. You checked the source, decided the work was finished, and moved the result back into the right file.

The model participated. You operated every connection around it.

What changes when AI stops being something you converse with and becomes something your software can call?

The answer is not “it gets faster.” The answer is that every one of those connections becomes an engineering responsibility with a name, a failure mode, and a place to put a test.

The chat box is an API with a person in the middle

A text box is a human-operated API. The input field accepts a request, the output field returns a result, and a person translates between that exchange and the application that actually needs the work done.

There is nothing disgraceful about this arrangement. For a single question, a person is the best integration layer. They resolve ambiguity without a schema and notice when an answer is beside the point.

But run the paragraph review across a whole manuscript and the hidden work surfaces as questions nobody can answer. Which paragraphs have been reviewed? Which version did the model see? Did the last answer refer to the current paragraph or the one before the edit? Was the revised sentence saved, or is it still in a scroll buffer? If the application closes, where does the work resume?

Copy and paste moves bytes. It leaves task identity and state in the operator’s head, where it cannot be queried, tested, or resumed.

The obvious response is to get better at prompting. That response has been studied, and it does not scale the way people expect. Zamfirescu-Pereira and colleagues gave ten participants without significant prompt-design experience a no-code tool for building an instructional chatbot on GPT-3, then watched how they worked. The participants explored prompt designs opportunistically rather than systematically: they overgeneralized from a single success, treated one good output as evidence that a prompt was robust, and wrote prompts shaped by expectations imported from instructing another human (Zamfirescu-Pereira et al., 2023). The authors connect these struggles to the long-documented difficulties of end-user programming and interactive machine learning.

Bound that result honestly. Ten participants, one task, one model generation, non-experts by design. It does not show that skilled engineers cannot write good prompts. What it shows is more specific and more useful: when the only available structure is a text box, people do not spontaneously invent evaluation, controls, or regression testing. The interface offers no place to put them.

Structure beat a better prompt

The complementary result is the more interesting one, because it changed the artifact rather than the operator.

Wu, Terry, and Cai took complex tasks that a single large prompt handled poorly and decomposed them into chains: sequences of small LLM operations where the output of one step becomes the input of the next, with every intermediate result exposed and editable. In a 20-person study, chaining improved the quality of task outcomes and significantly increased transparency, controllability, and users’ sense of collaboration (Wu et al., 2022).

The behaviors participants invented matter more than the quality delta. They used sub-tasks to calibrate what they could expect from the model. They compared strategies by watching parallel downstream effects. And they debugged unexpected output by “unit-testing” individual sub-components of a chain.

Read that last one again. Given intermediate results as inspectable objects, users spontaneously reinvented unit testing. Not because they were told to, and not because the model improved — the model was identical. The structure around it changed, and engineering practice became possible.

That is this book’s thesis in an early, modest form. The quality you get out of a model is bounded less by the model than by whether the surrounding process has anywhere to put a check.

Bound this one too: 20 participants, an interactive research prototype, GPT-3-era models, self-reported transparency and control alongside task quality. It demonstrates that decomposition gives people leverage. It does not prove chains are more accurate than monolithic prompts in general, and a badly-cut chain can be worse than one good prompt.

The first boundary

So we stop optimizing the text box and make the operation callable. At the client boundary, the process narrows to this:

explicit prompt → HTTP request → provider payload → CallResult

Here is a constructed teaching example using CodeAI’s actual adapter interface. The paragraph and the review task are illustrative; this is not a captured production run.

import uuid
from codeai.context import ContextCompiler
from codeai.domain import ActorRef, CallSpec
from codeai.providers import OpenCodeCognitionAdapter

paragraph = "The service processes every request within one second."
actor = ActorRef(actor_id="reviewer", kind="model", provider="opencode", model="mimo-v2.5")
package = ContextCompiler().compile(
    task_id="review-para-014",
    actor=actor,
    prompt=paragraph,
    prompt_version="claims-review-v1",
)
adapter = OpenCodeCognitionAdapter(
    model="mimo-v2.5", protocol="chat_completions", timeout=60.0,
)
spec = CallSpec(
    call_id=str(uuid.uuid4()),
    task_id="review-para-014",
    actor=actor,
    context=package,
    idempotency_key=str(uuid.uuid4()),
    instruction=(
        "Identify factual claims that require evidence. "
        "Do not invent sources. Return a short review."
    ),
)
result = adapter.invoke(spec)

if result.status != "succeeded":
    raise RuntimeError(result.error)
review = result.raw_output

This assumes the CodeAI checkout is installed and an OPENCODE_ZEN_API_KEY is configured. The model identifier and its chat-completions route come from CodeAI’s inspected route table, not a promise of gateway availability. Chapter 11 examines the recorded call without requiring a live service.

Two things changed and one did not.

The program now holds a value named review that it can store, hash, or pass onward. And it can tell that the request failed instead of reading a diagnostic sentence as though it were a review — the single most common integration defect in the wild, and the cheapest to fix.

What did not change: nothing has been verified. A generated review is a proposal about where to look. No amount of transport correctness establishes whether the service actually meets its latency promise.

From memory to fields — a constructed teaching example

The paragraph above names seven operator jobs (choosing, remembering, supplying criteria, distinguishing, checking, deciding, moving). Each becomes one stored field; the record below is illustrative, not a captured run:

import hashlib

paragraph_sha = hashlib.sha256(paragraph.encode()).hexdigest()
record = {
    "paragraph_id": "ch3-para-014",
    "paragraph_sha": paragraph_sha,
    "prompt_version": "claims-review-v1",
    "model": "mimo-v2.5",
    "review": review,          # content channel only; never an error string
    "disposition": "pending",
}

After this, the reader can do one new thing: hash any paragraph, attach the review to that hash rather than to memory, and refuse to file a review whose paragraph_sha no longer matches the file. That is task identity in its smallest form.

CodeAI’s OpenCodeCognitionAdapter makes this boundary unusually easy to see. Its invoke() builds a one-message request from the CallSpec, posts it with the configured timeout, extracts the text, and returns a CallResult carrying the output, the usage with its source label, and the failure in its own status channel. It does not retain a manuscript or append to a conversation history.

That is a source-level observation about one inspected adapter. It does not imply the remote service stores nothing, or that the rest of CodeAI is stateless. The point is ownership: this object does not own the review process. Which forces the useful question — what does?

Intelligence inside a process

The working definition for this book:

A model gives you intelligence. Applied AI is the engineering required to make that intelligence participate reliably in a process.

By the end of the book the same idea has a sharper form: the model is not the process. It is one selectively invoked component inside a process you can inspect.

“Reliably” needs a scope, or it is marketing. It does not mean every model answer is correct. It means the application can distinguish outcomes, retain what happened, control effects, and obtain the evidence its next decision requires. Some tasks should end unresolved. Some should go back to a person. A dependable process reports either outcome without pretending the objective was met.

The destination therefore contains a great deal more than generation:

    flowchart TD
    S["software state"] --> C["context assembly<br/><i>name exactly what the model sees</i>"]
    C --> M["model call<br/><i>the only stochastic step</i>"]
    M --> R["raw result<br/><i>preserved before interpretation</i>"]
    R --> E["evaluation<br/><i>what does this support?</i>"]
    E --> D["decision<br/><i>recorded against claims</i>"]
    D --> A["action<br/><i>authorized, bounded</i>"]
    A --> V["verification<br/><i>independent observation</i>"]
    V --> S2["new software state"]
    S2 -.-> S
  

Read it as a sequence of obligations, not a framework to implement this afternoon. Context assembly must name what the model receives. Raw result must survive a change of parser. Evaluation must decide what the output supports. An action must be authorized, verification must observe what actually happened, and new state must record enough to continue.

Not every arrow fires on every pass. A review may produce no action. A deterministic check may remove the need for another model call entirely. A person may need to clarify the objective. Later, a scheduler will choose among these operations — and the nature of that scheduler is the second of this book’s two bets.

The two bets

Most of this book is ordinary engineering. Two claims are not, and it is fairer to state them now than to spring them later.

Bet one: exactly one box in that diagram is allowed to be stochastic.

Look again. Context assembly, preservation, evaluation against recorded criteria, authorization, verification, state transition — every one of those has an exact right answer that a model is not needed to produce. Only the model call generates something you could not have enumerated in advance. The working rule, argued properly in Chapter 3, is that a component belongs on the deterministic side unless it demonstrably cannot be, and that if you are in any doubt about which side it belongs on, that doubt is itself evidence it belongs on the deterministic side.

And the stochastic box is not a regrettable necessity to be shrunk to zero. Its unreliability is the product. A deterministic function cannot propose a reading you failed to anticipate; that is precisely what you are paying the model for. The goal is not to eliminate stochasticity but to confine it — one box, with a preserved record of what went in and what came out, and a deterministic verifier on the other side.

Bet two: the thing that decides how to use intelligence must not itself be intelligent.

This book builds toward a router: the component that decides, for a given piece of work, which operation to run and which model to run it against. The claim is that the router’s own decision path contains no model call. If the process that governs the use of a stochastic component is itself stochastic, you have not solved the reliability problem — you have added a layer to it and made the failure harder to locate.

The stronger form, which we will have to earn: the router should not really be code either. It should be a specification — routes, budgets, authorities, and required checks as data that a small deterministic runtime interprets. Then a change in policy is a diff in a file you can read, review, and roll back, rather than a change in a program. Whether that ambition survives contact with the implementation is an open question in this book, and the answer will be reported either way.

Two implementations, one construction story

The project you will build is CodeAI. OpenCode is the primary model gateway for the walkthroughs. Cost is part of the motivation — if the process is going to make many small calls, the per-call price is an architectural constraint, not an accounting detail — but gateway pricing and route availability get verified during live labs rather than advertised here.

You will get prompts for the construction steps, not only finished architecture diagrams. The pattern is constant: inspect the existing project, request one bounded increment, run it, check the observed result, keep the evidence. Those prompts are written to work with Codex and Claude as well; the engineering obligations do not change with the assistant.

CodeAI supplies both the early model boundary and the runtime that comes after. Its OpenCode cognition adapter exposes the practical details that large abstractions hide: different request bodies, different usage fields, empty output, and errors that have to survive the trip back to the caller.

Its runtime supplies what comes after the single call. Its source separates cognitive calls, actions, and checks; it records events in an append-only SQLite ledger, stores artifacts by content hash, and represents claims with explicit evidence levels. These give us something concrete to examine when a single call stops being enough.

The adapter and the runtime are not presented as one already-integrated application. We move from the adapter’s transport boundary to the runtime deliberately, naming the contract at each transition. Where this book proposes a stronger contract than the code currently implements, it says so.

Later labs transfer the same pattern to a browser extension and an Android app. Those are planned builds, not results claimed here. The surfaces differ; the obligations — what was requested, what returned, and how completion is checked, including what may change — do not.

There is a reason CodeAI is the starting point. It was not bought; it grew, one missing capability at a time, around one author’s way of working. That used to be an unaffordable way to get software, and it is becoming an ordinary one. The chat window will not disappear from your day, and it should not. What should leave it is the process, and Chapter 30 argues that the process this book builds is what makes tools shaped around your own work trustworthy.

Failure modes

• Reading an error as content. A diagnostic string assigned to review and filed in the manuscript. Solved at the client boundary in Chapter 11.
• Associating a result with the wrong input. A delayed response attached to an edited paragraph. Needs task identity (Chapter 11) and an explicit context package (Chapter 15).
• Treating a suggestion as an authorized action. “Add a citation here” silently becoming a file write. Needs the capability/authority split (Chapter 20).
• Mistaking a successful request for a completed task. HTTP 200 is a transport fact, not a work outcome.
• Generalizing from one good output. The documented non-expert failure above, and it does not spare experts. One passing run is a sample, not an estimate.

What this chapter established

• In a chat workflow a person routes context, holds state, judges output, and transfers effects. Calling the model directly makes those responsibilities available to software; it does not discharge them.
• The difficulty is structural, not a skill deficit: with only a text box, there is nowhere to put a control, a check, or a regression test (Zamfirescu-Pereira et al., 2023).
• Decomposing one prompt into inspectable steps improved outcomes and let users invent unit-testing of sub-components without being asked (Wu et al., 2022).
• The definition we will hold to scope: a dependable process distinguishes outcomes, retains what happened, controls effects, and can report an unresolved result honestly.
• Two bets: only one component may be stochastic, and the router that governs it must not be.
• Obtaining a review value would demonstrate callable generation. It would not demonstrate factual verification, safe editing, restartability, or useful model selection. No live model outcome is claimed in this chapter.

Next

Before any of this is worth building, there is a prior question: why should you be the one building it? That diagram assigns the human intent, authority, and verification — plus judgment about when the model is the right tool at all. The next chapter argues that those four are the only positions in the picture worth holding, states plainly that this book will be wrong about some things, and shows what being wrong responsibly looks like using a study that reversed itself.

Continue with Never Stand in Front of the Steamroller.

References

• J.D. Zamfirescu-Pereira, Richmond Y. Wong, Bjoern Hartmann, and Qian Yang. Why Johnny Can’t Prompt: How Non-AI Experts Try (and Fail) to Design LLM Prompts. Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (CHI ‘23), Article 437, 21 pages. https://doi.org/10.1145/3544548.3581388
• Tongshuang Wu, Michael Terry, and Carrie Jun Cai. AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Prompts. Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (CHI ‘22), Article 385, 22 pages. https://doi.org/10.1145/3491102.3517582

Implementation source: CodeAI, src/codeai/providers.py (OpenCodeCognitionAdapter.invoke), src/codeai/domain.py (CallSpec, ActorRef), src/codeai/context.py (ContextCompiler.compile). Source identities and limitations are recorded in docs/applied-ai/evidence-map.md in the book repository.