← Distillation From First Principles

Coding and Structured Generation

Can a student learn to generate functionally correct structured outputs from teacher demonstrations rather than merely copying syntax? Constrained sequence likelihood, schema validation, test-based verification, and a bounded E3 study where verification filtering recovers the teacher-demonstration gap.

Chapter 24 ranked documents. The output was an ordering, and the evaluation asked where the relevant document landed. This chapter changes the output object again. A program, a SQL query, or a JSON document is not liked or ranked; it runs, parses, or fails. Its correctness is a property of the artefact, checkable without asking any model for its opinion:

Can a student learn to generate functionally correct structured outputs from teacher demonstrations rather than merely copying the teacher’s syntax?

On our bounded study, the answer is yes with a qualification that matters. Students trained on teacher demonstrations underperform students trained on independently correct targets on every seed (0.367 against 0.391 mean functional correctness). Verification filtering — keeping only the teacher outputs that pass an independent check — recovers the gap on the mean (0.402), though it trails the oracle-target arm on one of the three seeds. Syntax is essentially perfect in all arms, so the difference is functional, not formal. The student that copies the teacher learns the teacher’s shape; the student that copies only the teacher’s checked outputs learns the task.

Code as conditional sequence modelling

A code or structured-output model is, mathematically, the same object as the instruction models of Chapters 17–22: a conditional distribution \(p_\theta(y \mid x)\) over output token sequences \(y\) given a prompt \(x\), factorised autoregressively:

\[ p_\theta(y \mid x) = \prod_{t=1}^{|y|} p_\theta(y_t \mid x, y_{ What changes is not the algebra but the evaluation. A free-form completion is judged by resemblance — token accuracy, BLEU, human preference. A structured output is judged by a procedure: parse it, validate it against a schema, execute it against tests. The procedure does not grade on a curve. A JSON document with a missing brace is not "almost valid"; it does not parse. A function that returns the length of a list instead of its sum is not "close"; it fails the unit test.

This is why the chapter distinguishes three nested properties of an output. Syntax asks whether the tokens form a well-shaped artefact in the grammar (balanced braces, known tokens, terminated). Schema asks whether the shape carries the expected fields in the expected places (the right keys, digits where digits belong, no leading zeros). Functional correctness asks whether the artefact means the right thing (the sum is actually the sum). Each implies the previous one is satisfiable, but none implies the next. A model can master the first two and fail the third on every input — which is exactly what our students do on the inputs they get wrong.

Structured versus free-form outputs

Example A. The task this chapter measures. The prompt encodes three integers, say \([3, 1, 2]\), as tokens [BOS, INST_JSON, D3, D1, D2, EOS]. The correct completion is the JSON document {"sum": 6}, encoded as [LBRACE, K_SUM, COLON, D6, RBRACE, EOS]. The full row is these sixteen tokens padded to length 32. The input space is all tuples of 2–5 values in 0–9: 111,100 possible prompts, of which each seed uses 800 for training, 200 for validation, and 400 held-out for testing, pairwise disjoint by construction.

The output vocabulary has twenty tokens: the shared specials, the JSON punctuation, two key tokens, and ten digit tokens. There are no strings, no nesting, no comments, no imports. This is deliberately the smallest task that still separates the three properties: the template ({sum: __}) exercises schema, the digit sequence exercises syntax, and the addition exercises function. A sum of 6 and a sum of 35 differ only in digit count; the grammar accepts both, and only arithmetic tells them apart.

Contrast this with free-form generation, where the same prompt might accept “the sum is six”, “6”, “{"sum": 6}”, or a paragraph of working. Free-form evaluation must decide which of these count, and that decision is itself a judgement call. Structured evaluation outsources the decision to the grammar and the evaluator: the output either parses, validates, and computes correctly, or it does not. The chapter’s experiment uses the constrained form precisely so that no judgement call stands between the student’s output and its score.

Grammars, schemas, and the projection view

A grammar is a set of admissible token sequences. For our JSON-sum task, the admissible completions are exactly the strings matching

LBRACE K_SUM COLON digit+ RBRACE EOS

with the further restriction that the digit sequence has no leading zero unless it is the single digit zero. Call this set \(\mathcal{G}\). Unconstrained decoding samples from \(p_\theta(\cdot \mid x)\) over all sequences; grammar-constrained decoding projects onto \(\mathcal{G}\), typically by masking inadmissible tokens at each position and renormalising:

\[ p_\theta^{\mathcal{G}}(y_t \mid x, y_{ Our students do not use constrained decoding; they are trained to produce valid JSON and measured on whether they do. That is a harder test than masked generation, and the result — syntax validity 0.998 across arms — shows the template is the easy part. The projection view still matters conceptually: it separates "knowing the shape" (masking gets this for free) from "knowing the content" (no mask supplies the sum). A student that needs the mask to be valid has learned the distribution but not the grammar; ours learned the grammar and stumbled on arithmetic.

Schema validation is the executable form of the grammar. The implementation checks, in order: terminal EOS present, braces balanced, the K_SUM COLON header template, a non-empty digit run with no leading zero, and no unknown tokens. Each failure maps to an error category — syntax for unparseable outputs, schema for well-formed tokens in the wrong template, semantic for a valid document with the wrong sum.

A worked projection makes the masking concrete. Suppose at some position the student’s raw distribution is 0.50 on D6, 0.30 on RBRACE, 0.15 on K_SUM, and 0.05 spread over digit tokens, and the grammar state admits only digits (we are inside the number field). Masking zeroes the inadmissible mass (0.30 + 0.15 = 0.45) and renormalises the rest: D6 becomes \(0.50/0.55 \approx 0.909\) and the remaining digits share \(0.05/0.55 \approx 0.091\).

Masking makes the inadmissible impossible, not the student smarter

Figure 1. The worked masking step drawn as bars: 0.45 of raw mass removed, D6 renormalised to 0.909. Worked arithmetic, not a measurement. The projection did not make the student smarter; it made the inadmissible outputs impossible. Any measured gain from constrained decoding is therefore a property of the mask, not evidence the student learned arithmetic — which is why our students are evaluated without it. The taxonomy is not decoration: it tells the reader where the student’s capacity runs out. Here, the answer is unambiguous. Across 3,600 held-out predictions (9 students × 400 tests), invalid-output rates sit at 0.003; schema validity rounds to 1.00. Every failure that matters is semantic: a valid {"sum": N} with the wrong \(N\).

Why exact match is a weak proxy

Exact token-match accuracy asks whether the student’s completion is token-identical to the oracle’s. Functional correctness asks whether it parses, validates, and computes the right sum. On this task the two coincide numerically — canonical formatting (no leading zeros, fixed key order) means there is exactly one correct token sequence per input, so every functionally correct output is exact and vice versa. The manuscript reports both anyway, because their coincidence is a property of this grammar, not a general identity.

In general, exact match understates functional correctness whenever multiple outputs are correct. A Python function that sums a list can use a loop, sum(), or recursion; {"sum": 6} and {"sum": 6 } differ by whitespace but compute identically (our grammar forbids the second, but real JSON permits it). A student that finds a different correct program scores zero on exact match and full marks on tests. Conversely, exact match overstates correctness whenever the test set leaks into training: a student that memorised the test rows scores perfectly without computing anything. Both directions motivate the same discipline: report exact match as a diagnostic, functional correctness as the primary metric, and the train/test overlap audit as the precondition for believing either.

Test-based verification generalises the point. HumanEval (Chen et al., P-057) evaluates code models by running generated programs against unit tests rather than comparing tokens — the preprint that introduced Codex and its HumanEval set frames functional correctness, not resemblance, as the measure of code generation. The chapter uses that design as background only: HumanEval is a benchmark and a model report, not a distillation study, and the register records its reuse and contamination risks. Nothing about Codex’s scores is cited here; no number from that paper appears as evidence. What transfers is the evaluation idea — an independent check the teacher does not grade — which our integer evaluator implements in miniature.

Teacher demonstrations and behaviour cloning

A teacher demonstration is a prompt paired with the teacher’s greedy output for that prompt, right or wrong. Training on demonstrations is behaviour cloning: maximum likelihood on the teacher’s behaviour, \(L = -\log p_\theta(\hat y \mid x)\) where \(\hat y\) is what the teacher produced. There is no reward, no ranking, no filter. The student learns the teacher’s distribution including its errors.

Three properties of demonstration learning matter before any experiment. First, the student inherits the teacher’s error rate as a ceiling on its training signal: if 3% of demonstrations are wrong, the best the student can do by pure imitation is bounded by that noise. Second, the student’s errors compound differently from the teacher’s: the teacher’s mistakes are scattered across its own competence frontier, while the student’s mistakes concentrate where its smaller capacity gives out — here, multi-digit sums. Third, the teacher’s agreement with the student measures imitation fidelity, not task success. A student that reproduces the teacher’s wrong sum scores one for agreement and zero for correctness.

Verification filtering breaks the first property. Instead of training on all demonstrations, train only on the ones that pass the independent check — here, the teacher outputs whose JSON parses and whose sum matches the oracle. The filter needs no model, only the integer evaluator both teacher and student are measured against. It is the direct descendant of Chapter 19’s verifier-guided selection, applied to structured outputs rather than reasoning traces. The cost is data: filtering discards the teacher’s wrong outputs (2–3% here), shrinking the pool. The question is whether cleaner data beats more data at matched update budgets.

    flowchart LR
  T[teacher demonstrations] --> V{verify: parse + oracle check}
  V -->|pass| F[train on filtered set]
  V -->|fail| X[discard with reason]
  T --> B[train on all: behaviour cloning]
  

Figure 2. Arm B imitates everything; arm C trains only on what the independent check passes. Same teacher, same budget — the filter is the only difference.

The behaviour-cloning objective behind all three arms is worth writing down, because its gradient explains why the 3% matters. Given a training distribution \(q(x, \hat y)\) over prompts and demonstration outputs, cloning minimises the negative log-likelihood

\[ \mathcal{L}_{\mathrm{BC}}(\theta) = -\mathbb{E}_{q}[\log p_\theta(\hat y \mid x)] = -\mathbb{E}_{q}\left[ \sum_{t=1}^{|\hat y|} \log p_\theta(\hat y_t \mid x, \hat y_{ The gradient for one demonstration is \(-\nabla_\theta \log p_\theta(\hat y \mid x)\): probability mass moves toward the demonstrated tokens regardless of whether the demonstration is correct. There is no term that downweights a wrong \(\hat y\); the objective cannot distinguish a teacher's success from its mistake. If a fraction \(\epsilon\) of demonstrations are wrong, roughly an \(\epsilon\) share of gradient steps teaches errors. One hypothesis, not tested here, is that a student with less capacity than the teacher generalises those errors more broadly than the teacher made them.

On seed 50, \(\epsilon\) is about 0.03 (23 of 800 teacher rows wrong), yet B trails A by 0.050, more than \(\epsilon\) alone would suggest; amplification is one hypothesis. Filtering to \(\epsilon = 0\) raises C above B on every seed, though on seed 50 C still trails A (it trains on fewer unique rows, 777 against 800).

Example B — a small code task with unit tests

The JSON experiment is the measured instance; the code case is the worked illustration the contract requires, presented without training claims. Consider the specification: “write total(xs), returning the sum of a list of integers.” The tests:

assert total([]) == 0
assert total([3, 1, 2]) == 6
assert total([-1, 1]) == 0
assert total([10, 20, 30]) == 60

Four candidate student outputs separate the properties. Candidate 1, def total(xs): return sum(xs), passes. Candidate 2, def total(xs) return sum(xs) (missing colon), is a syntax error — it does not parse and no test runs. Candidate 3, def total(xs): return len(xs), parses cleanly and returns 3 for [3, 1, 2]; it is syntactically valid, type-correct, and wrong on every non-trivial test. Candidate 4, def total(xs): return 6, passes the second test and fails the rest — the memorised answer, functionally correct on one input and evidence of nothing.

Candidate 3 is the chapter’s central warning in miniature: the most dangerous student output is the one that looks most competent. It runs, returns an integer, and never raises. Only the tests catch it. Replace “unit tests” with “the integer evaluator” and “Python function” with “JSON document” and this is exactly the E3 study below, where every consequential error is a valid {"sum": N} with the wrong \(N\).

Example C — the leakage case

Suppose the test prompt [3, 1, 2] also appears in training. A student that memorised its training rows outputs {"sum": 6} correctly without adding anything. Exact match is 1, functional correctness is 1, and the measurement means nothing. The defence is the overlap audit: before any training, the runner checks that the train, validation, and test input-tuple sets are pairwise disjoint and refuses to train on any overlap. All three seeds pass with zero overlapping tuples out of 800/200/400. The audit is cheap, automatic, and non-negotiable — the same discipline Chapters 12 and 13 apply to teaching tasks generally. A structured benchmark without it is a memorisation test wearing an evaluation costume.

Distribution shift deserves one paragraph of scope discipline. Our held-out split is an input split: test tuples are unseen combinations of seen values and seen lengths, not new values, new lengths, or new templates. A student that memorised single-digit sums would still fail two-digit ones; our test set contains both, but both come from the same generator. Generalisation to longer inputs, larger values, or a changed schema is not measured and not claimed.

The frozen E3 comparison

The corpus is self-created: all 111,100 tuples of 2–5 values in 0–9, from which each dataset seed draws disjoint 800/200/400 splits. The teacher is a width-32, 2-layer transformer (27,776 parameters) trained 1,500 optimiser updates on oracle rows; its held-out functional correctness is 0.890, 0.910, and 0.915, clearing the 0.50 calibration gate on all seeds. The students are width-16, 1-layer transformers (4,464 parameters, a 6.2-to-1 parameter ratio), one per arm per seed, each trained EXACTLY 600 updates with identical initialisations per seed.

Arms differ only in training rows: A on oracle completions, B on the teacher’s greedy completions, C on the functionally-correct subset of the teacher’s completions (777, 792, and 783 rows — filter rates 0.971, 0.990, 0.979). Validation selects each arm’s checkpoint by its own hard cross-entropy after the full budget; the test set is evaluation-only.

Per-seed held-out results (exact token match equals functional correctness under canonical formatting; both are reported):

Seed Arm Exact / Functional Syntax Schema Agreement Invalid
50 A-oracle 0.393 0.998 0.998 0.430 0.003
50 B-teacher 0.343 0.998 0.998 0.380 0.003
50 C-filtered 0.352 0.998 0.998 0.380 0.003
51 A-oracle 0.360 1.000 1.000 0.368 0.000
51 B-teacher 0.350 1.000 1.000 0.352 0.000
51 C-filtered 0.388 1.000 1.000 0.405 0.000
52 A-oracle 0.420 0.993 0.993 0.435 0.007
52 B-teacher 0.408 0.995 0.995 0.427 0.005
52 C-filtered 0.465 0.995 0.995 0.470 0.005

Aggregates (mean ± SD, descriptive, not significance tests): A 0.391 ± 0.030, B 0.367 ± 0.036, C 0.402 ± 0.058 functional correctness; agreement 0.411 / 0.387 / 0.418; invalid rates 0.003 throughout.

Cleaner data beats more data at matched budgets

Figure 3. Mean functional correctness per arm: teacher demonstrations trail oracle labels on all seeds; the verified-filtered set recovers the gap. Frozen E3 measurements, descriptive means over 3 seeds. Token-level accuracy sits near 0.91 with NLL near 0.24 in all arms — the models predict most tokens well and fail at the arithmetic digits, which greedy decoding then gets wrong wholesale.

Three readings, each tied to a hypothesis. H1 holds on all seeds: oracle targets beat teacher demonstrations (+0.050, +0.010, +0.013). Copying the teacher costs correctness even when the teacher is strong (0.89+ validation) and its demonstrations are 97%+ correct; one reading is that the student amplifies the residual noise, though the experiment does not isolate that mechanism. H2 holds on all seeds: filtering beats raw demonstrations (+0.010, +0.038, +0.058) and lands at the oracle level (mean +0.011, mixed per-seed). Here, cleaner data beat more data at matched updates. H3 holds weakly: agreement exceeds functional correctness by about 0.02 in every arm. The overstatement is real but small — a reminder that agreement is the wrong metric, not a scandal.

Why agreement is not correctness, here concretely

For arm B on seed 50, teacher–student agreement is 0.380 and functional correctness is 0.343. The 0.037 gap decomposes into two populations: inputs where student and teacher agree and are both right (the bulk), and inputs where they agree and are both wrong (the teacher’s error reproduced). The second population is the interesting one: 3% of B’s training signal was wrong, and the student learned some of it faithfully. Agreement counts those faithfully wrong answers as successes. Functional correctness does not.

The filter’s effect is visible in the same decomposition for arm C. Removing the teacher’s wrong outputs removes the both-wrong population from training, and C’s agreement (0.380 on seed 50) now tracks its correctness (0.352) with the same small gap but a higher base. The lesson generalises beyond JSON: whenever the teacher is fallible, agreement rewards reproducing the failures. Only an independent check — tests, execution, arithmetic — separates the student that learned the task from the student that learned the teacher.

Cost, sandbox, and what was not run

Teacher training costs ~79–80 seconds per seed (1,500 updates); each student arm costs ~15.5 seconds per seed (600 updates); the study total is 385.5 seconds wall time with a 520.9 MB peak process high-water mark. Target-token exposure is ~134k per arm per seed. These are operation counts and wall-clock observations on one host, not prices; no API was used.

On sandboxes: the study executes nothing the model generates. The “execution” is integer addition and grammar checks in repository code over a closed token vocabulary — there is no eval, no exec, no subprocess, no generated Python. The sandbox question the contract raises is therefore answered by avoidance rather than by engineering: a constrained task representation removes the arbitrary-code hazard instead of containing it. A chapter that distils open-ended Python generation would need the enforced sandbox the contract describes; this one documents why its task does not, and states plainly that the safety argument does not transfer to executable code generation.

What was not run: no multi-digit multiplication (abandoned pre-run as unlearnable at these budgets — recorded in the protocol, not a result), no out-of-distribution values or lengths, no pass@k sampling (the evaluation is single greedy decode per input), no representation-similarity analysis, no external replication, no independent review.

Rights, licences, and contamination

The study uses RM-117, VERIFIED_PERMITTED for train_structured_distillation_e3. The corpus, teacher, and students are self-created; no external code, model, API, or dataset is used. A retrieval corpus is not the risk here — code is. Computer programs are protected as literary works under Directive 2009/24/EC, which permits observation, study, and testing but does not license training by default (L-027). Training on licence-encumbered code, including copyleft-licensed sources, can attach conditions to the resulting model; generated code can reproduce training code verbatim.

None of that occurred here — every token the students saw was generated from the integer grammar — but the record exists because the next structured task may not be synthetic. The HumanEval discussion above carries the same caveat: a widely reused benchmark can be contaminated, and contamination turns functional evaluation back into memorisation testing. The legal findings are dated and action-specific; they are not legal advice.

Limitations

One synthetic JSON-sum task, three seeds, tiny transformers. Sum-only outputs capped at 45; multiplication, multi-field schemas, nesting, and open-ended code are unmeasured. Single teacher budget, single student budget, single filter rule. The held-out split is an input split, not a distribution shift. Token accuracy near 0.91 coexists with functional correctness near 0.39 — the models are fluent and frequently wrong, and greedy decoding converts token uncertainty into total failure. No external replication and no independent review.

Preparing for tool use

Generating a correct output is a single-step decision: prompt in, artefact out, check against the specification. Chapter 26 removes that comfort. An agent must choose tools, read their observations, recover from mistakes, and produce a final answer after several decisions — each one a chance to compound the last error. The JSON evaluator becomes a tool environment; the greedy decode becomes a policy rollout; the error taxonomy becomes compounding-error analysis. The progression the batch promised — generating correct outputs, then choosing correct actions — starts with the machinery this chapter built: a deterministic environment, an independent success check, and the discipline of never grading the student against its teacher.

References

Chen, M., Tworek, J., Jun, H., Yuan, Q., Pinto, H. P. O., Kaplan, J. et al. (2021). Evaluating Large Language Models Trained on Code. arXiv preprint. arXiv:2107.03374. https://arxiv.org/abs/2107.03374

Directive 2009/24/EC of the European Parliament and of the Council of 23 April 2009 on the legal protection of computer programs. In force 5 May 2009. https://eur-lex.europa.eu/eli/dir/2009/24/oj (retrieved 12 November 2026). Cited as L-027 for the observation/study/testing context; not legal advice.