Jev From First Principles on Programmer.ie: Modern AI programming

Stop Generating

Mon, 05 Oct 2026 09:00:00 +0100

Every system that answers a question with a language model does the same thing: it asks the model to write some prose and then reads one value out of it. Sometimes the prose is a sentence, sometimes it is JSON, sometimes it is a single word followed by four paragraphs of explanation. The value is what the application wanted. The prose is packaging.

This chapter measures the packaging. Not philosophically — with a stopwatch and a token counter. The question is narrow on purpose: when a program needs one bounded value, what does it cost to obtain that value by generating prose? The answer is a number, and the number is the whole basis for everything the rest of this book tries to build.

The Jev Provocation

Mon, 05 Oct 2026 09:00:00 +0100

Chapter 1 ended with a question: the predicate cannot answer half the test split, so what is the thing that handles the rest? The vendor’s answer is a product. Before this book builds anything, it owes you a careful reading of that product’s contract — what it promises, what it pointedly does not promise, and which parts, if any, nobody had built before.

That last clause is the whole chapter. Everything Jev does arrives wrapped in the claim that it is a new kind of model. The claim might be true. It is more likely to be packaging. Packaging is not nothing — a good interface changes what gets built — but packaging and discovery are different achievements, and confusing them is how a book like this one lies to you. So: the contract first, from the vendor’s own pages only; then nine older mechanisms held up against it, one property at a time; then the verdict, including the one place where the verdict went against the prediction.

The Jev Controversy

Mon, 05 Oct 2026 09:00:00 +0100

Chapter 2 took the contract apart and found it was six-sevenths packaging with one candidate distinction. This chapter asks the adversarial question: granted the contract is mostly packaging, did Jev’s implementation beat the alternatives anyway? There is exactly one independent benchmark that can answer that, and this chapter reads all of it — including the parts that embarrass both sides.

You should know the shape of the answer before the evidence: Jev won one task outright and lost the other by three points, while a 200-million-parameter classifier sat three-tenths of a point behind the winner at a sixth of the latency. “Did not dominate” is the accurate reading. “Lost” is wrong. So is “won”. The rest of this chapter is about why those three readings keep getting confused, what each of the book’s frozen hypotheses predicts from here, and the measurement rules that will keep later chapters from repeating the benchmark’s own mistakes — because the benchmark makes mistakes, and finding them is part of the job.

The Smallest Decision

Mon, 05 Oct 2026 09:00:00 +0100

Chapters 1 through 3 argued about a product. This chapter builds the thing the rest of the book measures against: five deliberately boring providers, two real datasets, and the shared harness every later chapter imports. No new model, no hosted call, no cleverness. The question is narrow on purpose: how far do boring providers get, and with how few labels? Whatever a sophisticated provider achieves later has to beat this, by a stated margin, at a stated cost — or it has not earned its place.

Zero-Shot Decisions

Tue, 06 Oct 2026 09:00:00 +0100

Chapter 4 ended with a fixed world: 77 labels, frozen at training time, and a bar with a number on it. This chapter breaks that world open. In any deployed system the label set moves — new intents appear, policies get reworded, a product renames a category — and retraining a classifier every time is the tax the last chapter never priced. The question is what changes when the answer set is described in language at runtime instead of fixed at training time: what you gain, what you lose, and what the description itself costs you. If you have read Embeddings From First Principles you know the mechanism (label embeddings, cosine similarity); this chapter does not re-teach it. It measures what that mechanism is worth as a decision provider.

Decisions as Entailment

Wed, 07 Oct 2026 09:00:00 +0100

Chapter 5 opened the label set: 77 banking intents, 17 of them never trained on, and a provider that answers by embedding similarity. The question this chapter asks is older than that experiment. Is a “decision model” partly a rediscovery of natural-language inference? The idea is not new — Yin et al. (arXiv:1909.00161, §5) turned zero-shot classification into entailment a decade ago, and Wang et al. (arXiv:2104.14690, §4.3) showed that reformulating a task as entailment and fine-tuning lightly beats methods with 500× more parameters. The Red Hat benchmark used BART-large-mnli as its zero-shot baseline (research/cache/redhat-benchmark.md, Table 1–2), so this is also our first like-for-like contact with their ground. If a decision model is a new category, it has to beat the cheapest strong version of the old one.

The First Token

Wed, 07 Oct 2026 09:00:00 +0100

Chapter 4 fixed a world with 77 labels and put a number on it. Chapter 5 loosened one joint: the label set could be described at run time instead of trained in. Both chapters compared providers on accuracy. This one asks a question that comes before any provider runs.

Here is the shortest version of it. A decision model that returns a probability over a set of options must, somehow, turn “which of these?” into a number. The most common way is to write the options into a prompt and read the model’s distribution over whatever comes next. If the options are printed as letters, that is a probability over 26 possible answers. If they are printed as words, it is a probability over strings of different lengths. If they are printed as numbers, it is a probability over digits — and digits are not the same size in every tokenizer. The third-party account of how Jev works that this chapter set out to rebuild (research/primary-sources.md, Victor Dibia’s write-up — not vendor documentation; the vendor has published no internals) describes exactly this: a prompt that ends where the answer goes, then each option scored by its log-probability, then a softmax across options. That is the mechanism. It is also, as we are about to see, at least four different mechanisms depending on how the options are printed.

The Decision Contract

Wed, 07 Oct 2026 09:00:00 +0100

You replace the refund predicate with a classifier. The next ticket arrives, but your question names a label the classifier has never seen. Should the application guess, catch an exception, or pass the ticket to a human? A distribution over the wrong labels would look reassuring while answering a different question.

Stop Generating kept that application’s action separate from its provider. The Jev Provocation supplied a first request and response shape. The Smallest Decision and Zero-Shot Decisions then supplied unlike providers. The First Token exposed a further problem: some ways of presenting an option set cannot be read by a particular provider. You need to know whether it answered your request at all before asking how good its answer was.

Confidence Is Not Probability

Wed, 07 Oct 2026 19:00:00 +0100

You have a program that can approve a request or send it to a person. The provider returns a valid distribution, the selected option has a large probability, and your next line is if p > 0.9. What does that comparison buy you?

The Decision Contract checks that an answer covers the requested options and that inability is explicit. It cannot check how often a prediction is correct. You need observations for that second question. This chapter measures the available providers on CPU. It fits no calibration method; Calibration owns that work.

Calibration

Wed, 07 Oct 2026 22:00:00 +0100

You have a queue of requests and a program that branches on a provider’s score. Confidence Is Not Probability left you a warning: a valid distribution can be uninformative, badly scaled, or unreliable after the input distribution changes. You can collect labels and fit a correction. How many labels buy useful evidence, and what does your program need to remember?

The specification still defines the decision and its answer set. The provider still supplies scores. This chapter adds a fitted map and an evidence record. The runtime checks that they belong together. Your application chooses an action; neither a transformed score nor a provenance field chooses its consequences.

Abstain

Wed, 07 Oct 2026 23:45:00 +0100

Design draft: this chapter argues from the literature and from earlier measured results; its own experiment has not been run.

Your program branches on a provider’s score. Two chapters back, Confidence Is Not Probability showed you that the number is not a probability of being right, and Calibration gave you maps that fix that, inside the distribution they were fitted on. One option you have not yet given the program is the third answer: I will not act on this one. A decision that is never allowed to refuse must act on its worst guesses as surely as its best. Your queue of requests contains items your provider is confident and wrong about; calibration cannot remove them, it can only price them.

Unknown

Thu, 08 Oct 2026 09:00:00 +0100

Design draft: this chapter argues from the literature and from earlier measured results; its own experiment has not been run.

You have taught a program to refuse. Abstain gave the caller a typed way to decline a provider’s answer and a way to price that refusal. But one refusal shape is not enough. Consider three failures in a banking assistant. It sees a vague request to change a card limit and cannot choose between two nearly identical card intents. It sees a request to order paper checks when its menu contains only electronic banking actions. And it sees a refund instruction whose order ID is blank. The first is uncertainty among answers. The second is certainty that the answers are wrong. The third is not a decision at all: the mandatory state needed to decide is absent.

Reading the Hidden State

Thu, 08 Oct 2026 09:30:00 +0100

Design draft: this chapter argues from the literature and from earlier measured results; its own experiment has not been run.

You have been reading decisions from model outputs: letters, digits, brackets, full strings, entailment scores and calibrated probabilities. Those are all downstream of the same expensive object: a forward pass that turns a state into hidden activations, followed by whatever projection turns those activations into an answer. This chapter asks whether you should stop earlier. If the hidden state already contains the decision in linearly accessible form, a small probe may answer more accurately and more cheaply than the output readout. If it does not, or if the probe merely memorises its training labels, then hidden-state reading is an attractive illusion.

Tuning an Arbiter

Thu, 08 Oct 2026 10:30:00 +0100

Design draft: this chapter argues from the literature and from earlier measured results; its own experiment has not been run.

Up to now the book has mostly read decisions from frozen models. The First Token built an option-scoring provider without running its model sweep. Decisions as Entailment measured off-the-shelf NLI and found it losing to embeddings. Reading the Hidden State designed, but did not run, the probe comparison. This chapter asks the next question: if you are allowed to change weights, what is the cheapest change that makes a small model good at decisions?

Transfer Across Decisions

Thu, 08 Oct 2026 11:00:00 +0100

Design draft: this chapter argues from the literature and from earlier measured results; its own experiment has not been run.

Everything so far has been about one decision at a time. Zero-Shot Decisions showed that runtime-defined labels buy unseen-label capability at a price. Tuning an Arbiter designed, but did not run, the comparison that decides whether tuning earns its cost. This chapter asks the question those chapters were always pointing at: when you train on some decisions, do unseen decisions get better, or did you just learn twenty separate classifications?

The Decision Expression

Thu, 08 Oct 2026 12:00:00 +0100

Design draft: this chapter argues from the literature and from earlier measured results; its own experiment has not been run.

You have four chapters of machinery and no way to call it. The contract validates answers. The abstain gate prices refusal. The unknown resolver names three ways of not deciding. Each is a function you could call, but nothing says in which order, what happens when one step fails, or what the caller is allowed to ignore. So here is the failure this chapter exists to prevent, in the smallest form that still runs:

if decide

Thu, 08 Oct 2026 13:00:00 +0100

Design draft: this chapter argues from the literature and from earlier measured results; its own experiment has not been run.

The Decision Expression gave you a value that refuses to be ignored: an Outcome that is either a Decision or a typed non-answer. Now you have to use it, which means branching on it. And here is the failure this chapter exists to prevent, in the form every working programmer has written at least once:

Semantic Match

Thu, 08 Oct 2026 14:00:00 +0100

Design draft: this chapter argues from the literature and from earlier measured results; its own experiment has not been run.

if decide gave you two arms: value and uncertain. But decisions have more than two shapes. A choice among seventy-seven intents is not a boolean, and collapsing it to “answered or not” throws away the structure the caller actually branches on: which answer, how close the runner-up was, and whether the question was answerable at all. This chapter builds match decide, the K+1-armed generalisation, with one arm per listed option plus the mandatory uncertain arm. It asks what each arm means when scores compete on wording rather than meaning.

Decision Types

Thu, 08 Oct 2026 09:00:00 +0100

Decision Types

The problem

You have a decision to make. The answer could be one of several options, or a number on a scale, or a preference between two things, or a set of options. The vendor’s contract gives you three question types: Choice, Score, and Noul. But the programming language you are building needs to know which of these are genuinely different semantics and which are just representations of the same underlying thing.

Decisions Cannot Read Everything

Thu, 08 Oct 2026 09:00:00 +0100

Design draft: this chapter argues from the literature and from earlier measured results; its own provider experiment has not been run. What did run is a token count of a real corpus and a toy model whose output is labelled as the consequence of its assumptions.

You are building a decision system, and the evidence it needs lives in a corpus: documents, passages, records. The decision model can read only a finite amount of text. The contract you have been using hands the provider a single state string, so the easiest design is also the most tempting one: put the whole corpus in the state and let the model sort it out.

Embeddings as Candidate Generation

Thu, 08 Oct 2026 09:00:00 +0100

Embeddings as Candidate Generation

This chapter reports a CPU experiment that ran here: an open encoder (bge-small-en-v1.5) and BM25 over a small public dataset (SciFact), on the machine described in evidence/environment.md. It also shows a model-free walkthrough. The encoder is a relevance model, not a decision model: nothing here is evidence about Jev or about H1-H4.

The problem

Chapter 20 ended with a claim: a decision cannot read everything. If the state is larger than the model can use, the decision layer must be fed a smaller state. Something has to choose what it reads.

where decide

Thu, 08 Oct 2026 09:00:00 +0100

where decide

This chapter reports a CPU experiment that ran here: a cross-encoder (ms-marco-MiniLM-L-6) and an encoder (bge-small) over SciFact. A cross-encoder is a relevance scorer, not a decision model in the Jev sense. Nothing here is evidence about Jev or about H1-H4.

The problem

Chapter 21 measured a pipeline where retrieval found the evidence and the decision layer threw a third of it away. The decision layer was a thresholded embedding cosine, and it was weak. So the question is not whether to filter, but which filter, and does the filter deserve syntax?

for decide

Thu, 08 Oct 2026 09:00:00 +0100

for decide

This chapter reports a CPU experiment that ran here: the loop strategies over the Chapter 22 SciFact candidate lists and their cached cross-encoder scores. No language model was run in this chapter. The result is a rejection of the construct, which the chapter is allowed to make.

The problem

Chapters 21 and 22 showed that something must select what a decision reads, and that the filter is where the end-to-end loss lives on SciFact. This chapter asks the next question: is iteration its own construct?

Decisions About Decisions

Thu, 08 Oct 2026 09:00:00 +0100

Decisions About Decisions

Design draft: this chapter argues from the literature and from arithmetic. It runs no provider. Every number below is a consequence of a constructed contingency table or an assumed conditional table, and the assumptions are swept and stated next to the results. None of it is evidence about any model. The provider experiment is deferred.

The problem

Chapter 16 kept decide a pure function: it takes a state, asks a provider, returns a typed outcome. It declined to say how one decision feeds another. Chapter 22 showed why that matters: a filter decides which candidates a later decision reads, and its errors become the later decision’s errors.

Semantic Pipelines

Thu, 08 Oct 2026 09:00:00 +0100

Semantic Pipelines

This chapter reports a CPU run of the pipeline’s retrieval and relevance stages over SciFact, and the full pipeline with a declared weak relation provider. The relation (entailment) and generation providers need a language model and are deferred. The author’s private Writer evidence is off limits; the public claims are SciFact’s.

The problem

Chapters 21-24 built the pieces: candidate generation, filtering, relation, composition. This chapter asks whether they fit together into a real task. The task is claim research: given a claim, retrieve evidence, decide whether the evidence supports or contradicts the claim, produce an answer, and decide whether that answer is supported.

Decision Graphs

Thu, 08 Oct 2026 09:00:00 +0100

Decision Graphs

Design draft: this chapter builds the graph machinery and runs a fake-provider replay study on CPU. No model runs; every provider is a deterministic or seeded fake. The numbers measure the machinery, not any model.

The problem

Chapter 25 showed a pipeline whose every error is attributable to one stage. But a list of stages is a weak container for that attribution: it cannot say why a verdict depended on some evidence and not other evidence, and it cannot be replayed after the fact to check whether the world has moved.

Decisions Over Time

Thu, 08 Oct 2026 09:00:00 +0100

Decisions Over Time

Design draft: this chapter computes ANALYTICAL token counts under assumed costs. No model and no inference engine runs. Every number is a consequence of the assumed state length, question length, per-call overhead and batch-scaling exponent, which are swept and stated next to the results. None of it is a measurement of Prompt Cache, SGLang, or any engine.

The problem

A decision request in the vendor’s model is one state, several questions, answered in one call. The shared state is the selling point: ask several questions over one state, and the work is shared. Chapter 26 recorded the graph around this. This chapter asks the arithmetic question the claim rests on: do several questions over one state share work, and does that matter?

One Contract, Many Models

Thu, 08 Oct 2026 09:00:00 +0100

One Contract, Many Models

This chapter runs no new experiment. It compiles the winner table from rows in results/ch04.jsonl, ch05.jsonl and ch06.jsonl and marks every cell without a measured split=test row as NOT_OBSERVED. The number of filled cells (17) is smaller than the number of empty ones, and that is a result.

The problem

Chapter 8 defined the contract every provider satisfies. This chapter asks the question the contract exists for: where does each kind of provider win, on the same decisions?

The Decision Router

Thu, 08 Oct 2026 09:00:00 +0100

The Decision Router

This chapter is a simulation over assumed provider profiles. The provider accuracy-by-difficulty curves, costs and signal correlation are assumed and swept; every number is a consequence of them. No real router (RouteLLM, Hybrid LLM) is measured. The assumptions sit next to the results.

The problem

Chapter 28 ended with a table mostly full of NOT_OBSERVED. A router makes that honest gap useful: it chooses, per request, the cheapest provider that is competent for that request, so quality comes from where it is measured and cost is saved elsewhere. This chapter asks: can a router choose the cheapest competent provider for each request?

Cascades

Thu, 08 Oct 2026 09:00:00 +0100

Cascades

This chapter is a simulation over assumed stage profiles. No cascade is measured. Every number is a consequence of assumed accuracy curves, costs, confidence noise, shift and human capacity, all swept and stated next to the results. In particular: the naive product of stage accuracies does not equal the measured end-to-end accuracy, so any “cascades save X%” headline below is a statement about the swept assumptions, not about real providers.

Learning the Router

Thu, 08 Oct 2026 09:00:00 +0100

Learning the Router

This chapter is a simulation over assumed provider profiles. Every accuracy curve, cost, signal noise and audit fraction is assumed and swept; every number is a consequence of them. No real router (RouteLLM, LinUCB) is measured.

The problem

Chapter 29’s router was explicit rules over metadata. This chapter asks what happens when the router learns the answer from the outcomes it observes. The catch is that what a router observes is decided by the router itself. A router that only sees the outcomes of the providers it selects can be systematically wrong about the providers it avoids — and worse, its mistakes can be self-reinforcing.

The Decision Specification

Thu, 08 Oct 2026 09:00:00 +0100

The Decision Specification

This chapter is model-free: a declarative DecisionSpec and a validator, with six seeded bug classes and a reported detection rate. No model runs; the numbers are the validator’s own outputs.

The problem

Every chapter so far asked a provider a decision through the library API. The API checks a single call in isolation. But a decision in a program has a life beyond one call: it may abstain, and the abstained request must go somewhere; it may fall back to another provider; its answer classes must each have a consequence; its thresholds must be coherent with its risk target. None of that is expressible in a single library call, and none of it is checked by the call’s own validation.

The Decision Compiler

Thu, 08 Oct 2026 10:00:00 +0100

The Decision Compiler

This chapter is a simulation over supplied profiles. The provider curves, costs and the cascade’s end-to-end profile are seeded from earlier rows (Ch04’s measured LR; Ch30’s simulation rows) or are explicit ASSUMPTIONS. The compile-time table is arithmetic on those profiles; the runtime numbers are a SIMULATION over the same assumed curves. No new model runs.

The problem

A decision contract is now declarative: Chapter 32’s DecisionSpec names the inputs, answers, risk, abstention route and fallback chain. This chapter asks the next question: given the spec and a set of provider profiles, can a compiler pick the cheapest provider that satisfies the contract?

Semantic Programs

Thu, 08 Oct 2026 11:00:00 +0100

Semantic Programs

This chapter is model-free: the program runs over stub providers with hand-supplied numbers (labelled ILLUSTRATIVE) and deterministic retrieval (BM25, Chapter 21). The generation step is a stub. The ReAct-style comparison is a count over a declared task script — no agent is run (no LLM is allowed by the book’s rules).

The problem

The book has spent sixteen chapters building pieces: decide, if decide, match decide, where decide, the spec, the compiler, the graph. This chapter asks the question the pieces were for: what does a program with deterministic and semantic computation together look like, end to end?

Programming with Decisions

Thu, 08 Oct 2026 12:00:00 +0100

Programming with Decisions

This chapter is a verdict over the ledger, not an experiment: one mechanical extractor (run_ch35.py) reads the metadata preregistrations and writes results/ch35.jsonl; every number below is from that file or from a cited ledger row. No model ran. The hypothesis classification follows the author’s floor for Session E: no hypothesis is decided above INSUFFICIENT_EVIDENCE without a real measured provider.

The question

On the evidence, what is a decision model, and is decision a programming primitive? The book must answer from what it actually measured, and the answer is allowed to be narrower than the investigation set out to build.