Build a Fine-Tune Without a Training Framework
Assemble tokenisation, causal loss, gradients, updates and resource checks into a measured tiny source-training and saved-weight adaptation experiment.
A model trained from random weights can demonstrate learning without demonstrating fine-tuning. To observe adaptation, we need an existing learned state, a subsequent change in its training objective or distribution, and a comparison with what would have happened without that change. Saving a randomly initialised model and naming the next loop a fine-tune would miss that distinction.
We now assemble the components from Chapters 07–11 into two actual stages. First a small causal Transformer learns a synthetic source rule. Then we reload its saved weights and continue training on a related but conflicting rule. A source-continuation control begins from the same saved weights. The experiment is intentionally tiny enough to inspect completely; it uses no pretrained language-model weights or production incident records.
Choose a task whose answers can be audited
The input contains a letter from a through h followed by a context digit. The source rule returns the letter unchanged. The adaptation rule returns its cyclic successor: a becomes b, continuing through h becoming a. Both answers terminate with the educational ending token.
For example, the source answer to user text a4 is a; the adaptation answer is b. The context digit identifies an input variant and does not determine the answer. Its allowed values are divided before training:
| Context digits | Role | Cases per rule |
|---|---|---|
| 0, 1, 2 | Training | 24 |
| 3 | Validation | 8 |
| 4, 5 | Held-out evaluation | 16 |
The splits contain different complete input strings. All eight letters still appear in training, so this is a small compositional generalisation check over contexts, not evidence of independence across novel scenario families or languages. The complete synthetic manifest and label rules are written before any update, and their identity is bound into the run configuration and checkpoints.
The two rules deliberately conflict for the same inputs. A deterministic model given no task indicator cannot answer both a and b for the same a4 prompt. That design makes retention easy to interpret: a model that fully switches to rotation must lose correctness under the old copy rule. It is not an experiment discovering that forgetting always has this magnitude in realistic applications.
These educational records are separate from the incident-triage benchmark. Fixing their manifest for this exercise does not freeze the later production evaluation dataset or grant it any training permission.
Keep the representation consistent with the chosen model
The model uses the unmerged configuration of Chapter 07’s byte tokenizer: 256 ordinary byte entries and six educational control entries, for vocabulary size 262. Chapter 07 also trained a twelve-merge configuration to illustrate vocabulary construction. That inspection configuration has additional entries and a different tokenizer identity.
The shared Hi./OK worked batch uses only unchanged original byte/control IDs, which is why the loss and gradient exercises could consume its arrays directly. This does not make our 262-entry model compatible with every output of the learned-merge tokenizer. Supplying an out-of-range merge ID is rejected. Here the training tokenizer is explicitly unmerged, and its complete identity is bound to the model checkpoints.
The same compile_dialogue and make_batch components create each example’s IDs, attention mask and labels. They still supervise assistant content and its ending, and exclude the prompt and control header from targets. For our two-character prompts and one-character answers, complete sequences have eight positions and two supervised targets.
The main model is the same pre-normalised causal Transformer introduced earlier, now with width 32, two heads, MLP width 64 and context capacity 32. It has 26,400 parameters, with separate input embeddings and output weights. Dropout is zero in the main comparison. Parameter storage and calculations are FP32.
Make one optimiser step readable
The training loop has no high-level trainer. Its important operations are visible:
model.train()
optimizer.zero_grad(set_to_none=True)
mean_loss, target_count = accumulate_token_mean(
model, input_ids, attention_mask, labels, microbatch_size=4
)
clip_grad_norm_(trainable_parameters, 1.0, error_if_nonfinite=True)
optimizer.step()
scheduler.step()
The step samples eight training records, splits them into two four-record microbatches and preserves one total-token denominator. accumulate_token_mean calls Chapter 08’s loss, which owns the one causal shift. The model calls Chapter 09’s explicit block. The optimiser constructor and accumulation semantics are the checked components from Chapter 10. Storage reporting uses Chapter 11’s ledger.
Inside the shared accumulation function, the numerical work remains explicit:
for start in range(0, batch_size, microbatch_size):
stop = min(start + microbatch_size, batch_size)
logits = model(ids[start:stop], attention[start:stop])
terms = causal_terms(logits, labels[start:stop])
(terms["sum"] / total_target_count).backward()
Here stop is the end of the current microbatch and the denominator covers the
complete optimiser step. The actual helper calculates both bounds for each
iteration. It also detaches the loss used for logging, so a reporting list does
not keep an entire training graph alive. This is the same tested helper, rather
than a second implementation hidden inside the capstone.
flowchart TD
S["Load saved source weights\n+ manifest identity"] --> Sm["Sample 8 training cases\nseeded generator"]
Sm --> Cp["Compile IDs, masks, labels\nshared Ch07 batch"]
Cp --> Z["Zero grads once"]
Z --> A["Accumulate 2 x 4 microbatches\ntoken-weighted denominator"]
A --> Cl["Clip once · norm 1"]
Cl --> St["Optimiser step · scheduler step"]
St --> Sv["Commit checkpoint + receipt"]
Sv -->|"96 steps adaptation\nvs 96 steps continuation"| Ev["Evaluate: greedy generation\n+ teacher-forced NLL"]
The loop makes the step boundary and the two-branch comparison visible: adaptation and source-continuation start from the same saved weights and differ only in the labels they train on. Held-out answers are generated, never trained on, and the rotation/copy trade-off is reported on both sides.
Clearing occurs once before accumulation, not before every microbatch. Clipping occurs after accumulation, not separately for each contribution. The schedule advances after the optimiser step. Every completed step records its selected-target count, pre-update mean NLL, gradient norm before clipping and learning rate actually used.
The sampler has its own seeded CPU generator and draws with replacement from the declared training list. Its generator state and consumed-example count define the next draw at a resume boundary. The loop rejects records marked as validation or held-out. That is a mechanical safeguard around this trusted educational manifest, not a replacement for a production data-access authority.
The training function never receives the held-out list. Validation is reported without selecting new settings, and the step budgets are fixed in advance. We do not inspect a held-out result and then choose a more flattering learning rate or seed.
Separate initial training from adaptation
The source phase uses 160 optimiser steps, base learning rate 0.01 and weight decay 0.01. All parameters are trainable. Its schedule is the explicit four-step warm-up multiplier and bounded decay from Chapter 10. This creates the learned state that we subsequently adapt.
After saving that state, two new phases each use 96 steps, base learning rate 0.006, the same decay and matching new schedule settings. Both freeze token and position embeddings, leaving 16,992 trainable parameters. The adapted candidate uses rotation labels; the source-continuation control keeps copy labels.
Each new phase loads exactly the same saved source weights. Each intentionally creates a fresh optimiser and schedule. This is a controlled choice for adaptation and its comparison, not an assertion that moments have been restored from an interrupted source run. Resetting the optimiser changes the subsequent dynamics and is explicitly recorded.
The original source model is also retained as the no-adaptation baseline. The continuation control adds a further comparison: continued updates under the original rule versus the same budget of updates under the changed rule. Freezing policy, optimiser settings and sampled index sequence match between those two phases.
The main comparison therefore contains 352 optimiser steps in total: 160 source, 96 adaptation and 96 source control. The separate short recovery probe adds 40 updates across its uninterrupted and interrupted/restarted branches. All work remains inside the declared per-process budget of 512 steps and 100,000 parameters per model, with sequence length at most 32 and batch size at most eight.
Measure the answers the model actually generates
Teacher-forced loss scores the correct recorded history. It cannot substitute for observing what the model emits when it conditions on its own outputs. Our evaluator reports both quantities.
For generation, start with the serialised user message and assistant opening. Predict one ID greedily, append that prediction to the input, and predict again if an ending has not appeared. The limit is two new tokens because a correct answer here contains one letter plus EOS. Exact correctness requires the entire expected ID sequence, including that ending. A missing ending, an early ending or a wrong letter is a failure.
Teacher-forced NLL is computed separately over the two selected target positions per case. Evaluation uses model.eval() and no gradient recording, then restores the caller’s prior training mode. This separation matters even though the main comparison has zero dropout: the recovery test uses non-zero dropout, and future models can have other mode-dependent behaviour.
A negative test constructs a predictor that scores the teacher-forced answers well while generating the wrong letter from its own prefix. The evaluator reports low teacher-forced NLL but zero exact generated accuracy. That test protects the reporting boundary against quietly replacing autoregressive behaviour with an easier statistic.
Read the measured result within its small world
The CPU reference and bounded CUDA run produced these held-out exact-answer counts:
| Saved state or candidate | Copy rule, 16 cases | Rotation rule, 16 cases |
|---|---|---|
| Source-trained model before adaptation | 16/16 | 0/16 |
| Source-continuation control | 16/16 | 0/16 |
| Adapted model | 0/16 | 16/16 |
| Random-initialised model trained directly on rotation, independent review | 0/16 | 16/16 |
| Tagged nonconflicting control after adaptation, independent review | 14/16 | 10/16 |
The last two rows were separately executed during the independent Part II review. The random-initialisation control used 96 rotation updates; it reached the same exact rotation count without source training. The tagged control supplies different task contexts and tests a different, nonconflicting problem. These controls show that fine-tuning is neither established as necessary nor superior nor more sample-efficient here. The result demonstrates training mechanics and contradictory objectives, not general catastrophic forgetting.
These counts are actual greedy answer-plus-EOS results. The independent numerical audit reloaded the saved states and reconstructed every held-out output using a separate generation loop. It also recalculated NLL using scalar log-sum-exp rather than the training loss function. Reported and recomputed outputs agreed on both devices.
For the CPU run, mean held-out rotation NLL changed from approximately 3.894326 for the source model to 0.0091567 after adaptation. Mean copy NLL changed from approximately 0.0043507 to 2.684100. The two metrics describe the intended gain and the designed retention loss together, rather than selecting only the favourable column.
Recorded source minibatch NLL was approximately 5.767813 at its first step and 0.004439 at its last. Adaptation’s corresponding endpoints were approximately 3.778802 and 0.008593. These are losses on sampled training minibatches, which need not decrease monotonically or contain identical records. The held-out comparison above is separate.
The experiment demonstrates actual adaptation because its second stage starts from learned, saved weights and changes them under a new rule. It does not demonstrate useful Qwen fine-tuning, realistic language generalisation, safe deployment or an economic advantage over a rule system. A direct eight-letter lookup would solve these tasks cheaply; its purpose here is to provide a transparent numerical learning problem.
Preserve a recoverable learning state
A model state dictionary stores parameters and persistent buffers. That is sufficient for restoring the model’s function under the same architecture, but not generally for reproducing the next training update. The optimiser’s moments, schedule position and random data choices also influence that update.
The exercise checkpoint stores:
- model and optimiser state;
- scheduler state and completed-step count;
- sampler state and number of consumed examples;
- global PyTorch and Python RNG state, plus CUDA RNG state when used;
- tokenizer, template, data, model configuration and implementation identities.
The file is written to a temporary name and renamed after a completed save. A receipt records its content hash and committed step. Existing checkpoint destinations are refused. Before loading, the code checks the supplied trusted prior digest, then compares the expected identity before applying the state. The examples use restricted weights_only=True loading for their own trusted local files; this is not advice to execute arbitrary untrusted checkpoint content.
The identity comparison rejects a different tokenizer, data manifest, model shape, training phase or implementation. A matching file hash proves consistency with the expected bytes, not who produced them or whether the experiment’s claims are honest. The production execution and source-binding gates remain separate.
The main source checkpoint reloaded with zero maximum parameter difference. A stronger recovery probe uses dropout 0.1, interrupts after step 7 of a 20-step phase, restores all the recorded learning state and continues. The final model parameters and complete loss/learning-rate history matched the uninterrupted branch exactly on each tested device.
This is a same-phase, same-device, same-software result. It is not a promise of bit-identical execution across CPU and GPU, library versions or different accelerator kernels. The main loop is FP32, so it has no active loss-scaler state; an AMP resume path would also need to preserve its scaler. Additional data workers or random preprocessing would introduce further state to account for.
Stop at an honest boundary
An interruption between completed updates has a clear meaning: the last committed step is known. A process killed in the middle of an update cannot safely claim that an arbitrary in-memory snapshot represents a completed next step.
The bounded runner requests cooperative stopping through a flag checked at a step boundary. If requested or if the training deadline is reached, it saves a committed state and reports an incomplete experiment instead of marking the intended total steps as completed. An outer timeout provides a further bound. A partial save is not promoted into a completed checkpoint, and a hard kill can still require recovery from the most recent completed file.
The recovery test exercises that committed-step boundary. It does not simulate every operating-system failure, power loss, disk failure or distributed recovery case. Those broader operational concerns belong to the later recovery chapter.
Compare devices without changing the story
The CUDA run executed on the reference RTX 2060 in Docker and used FP32 for the main training. Its complete capstone function took approximately 12.842 seconds in the recorded final-code run and peaked at 69,650,432 allocator bytes. The corresponding CPU function took approximately 2.282 seconds. These intervals include the exercise’s preparation, training, evaluation and recovery work inside the runner, but exclude Docker startup and Python import time.
The GPU was slower for this tiny operation-rich model, just as the smaller windows in Chapter 11 suggested. We do not extrapolate that ordering to a large pretrained model. The useful result is that the same bounded learning experiment executed on both devices with recorded provenance.
CPU and GPU produced the same exact-answer counts, with small floating-point differences in NLL and parameter values. Their source-state content digests differ, so we do not call the weights byte-identical across devices. Separate CPU repetitions did reproduce the recorded parameter-state digests and training histories exactly. That is a narrower and better-supported reproducibility claim.
The stable Docker entry points are book-edu-cpu and, under the GPU profile, book-edu-gpu. Each chooses fresh output paths unless an explicit unused path is supplied. The runtime is offline and the project source is read-only; generated weights and checkpoints remain in the ignored results area. No model weights or credentials are baked into the image.
Keep the educational result separate from production admission
The pipeline now has every essential small-scale learning step in view: tokenisation, masking, a causal forward pass, a scalar objective, backward propagation, accumulation, an update, storage measurement, saved-state adaptation and generated held-out answers. Its parts are shared rather than replaced by unrelated demonstrations.
Several important questions remain deliberately unanswered. Our splits are tiny and synthetic. Their conflicting rules are constructed, their labels are unambiguous, and the model is trained from scratch before adaptation. We have not measured real-world sample adequacy, language retention, privacy risks or an adapter’s benefit over production baselines.
Nor have these experiments closed the production requirements for scorer authority, operational constraints, fresh resource admission, captured-source binding, execution-source identity and actual weight-file verification. The original incident-triage contract still forbids training and its evaluation dataset is not frozen. No Qwen training occurred.
Part III will replace the convenient assumptions of this miniature exercise with explicit dataset rights, labelling, representation and independence checks. The numerical loop is no longer mysterious; the next problem is whether the examples it learns from deserve the authority we are tempted to give them.
Further Reading
- Andrej Karpathy, “nanoGPT” (code repository). Read the source. The closest public analogue of the chapter’s framework-free loop, whose finetuning path of continued training from loaded weights matches the source/adaptation structure.
- James Kirkpatrick et al., “Overcoming catastrophic forgetting in neural networks” (2016). Read the source. The canonical continual-learning reference for the retention measurement, explaining why copy-rule loss must be reported alongside rotation gain.
- PyTorch contributors, “Reproducibility” (PyTorch 2.6 notes). Read the source. Authorises the narrow reproducibility claim: seeded same-device replay can match exactly, while cross-device bit identity is not promised.