← Fine Tuning From First Principles

Instruction, Completion and Conversation Datasets

Preserve semantic authority while rendering labelled records into deterministic completion, instruction and chat examples.

A labelled record can be perfectly valid and still be a disastrous training example. Put its answer in the user prompt and the model can learn to copy a supplied label. Omit the policy table and the model must guess which routing revision the annotator used. Duplicate a conversation turn during conversion and the intended response may appear twice. None of these failures requires invalid JSON.

The previous chapter established what an answer means. We now decide which information is supplied as context, which text is the desired response, and how those pieces become an ordered sequence. Representation is part of the experimental intervention because a model learns from tokens, not from our intention that two database rows mean the same thing.

We will compare completion, instruction and conversation forms, together with structured-output and multi-turn variants. All retain their original authority and provenance. Their byte identities differ, while a policy-independent semantic identity continues to identify the underlying incident.

Keep the canonical record outside the rendered prompt

A canonical record contains both data the application receives and metadata the experiment needs. Family IDs, label provenance and split assignments help audit an experiment. They do not automatically belong in the model input. Supplying family_id="security-critical" could make the answer recoverable without reading the incident. Supplying a split name could teach a convention that does not exist at deployment.

Our prompt contains only the task input and the applicable policy table. The input is already defined by the task contract. The annotation supplies the target output. Record IDs and source digests remain in the outer training-record envelope so the auditor can trace the example without teaching the model to use those metadata fields as evidence.

The input does contain its contract-defined incident_id. That field locates an incident but is explicitly prohibited as a label shortcut. Dataset construction should avoid category-encoding identifiers when creating later training studies. The present educational fixtures expose their lineage openly; they are not evidence that a model has ignored every possible identifier shortcut.

An active policy is legitimate context because the intended route depends on it. A target annotation is not legitimate context for ordinary prediction. This distinction is about the task’s information boundary, not whether both objects happen to be stored next to each other in a JSON file.

Rendering: what crosses the information boundary

The figure places each field on its side of the rendering boundary. Notice that provenance travels in the envelope beside the prompt so deletions and contamination traces stay possible, while family and split metadata must never enter the prompt itself.

Three formats describe three interfaces

An instruction example has an instruction/context string and response. A completion example has a prefix and the text that should follow it. A conversation has ordered role/content messages. These forms can express the same intended incident answer, but they do not guarantee identical tokenisation under every real model’s template.

For the access example under p1, the rendered content is schematically:

{
  "messages": [
    {"role": "user", "content": "{\"incident\":{...},\"policy\":{...}}"},
    {"role": "assistant", "content": "{\"abstain\":false,...}"}
  ]
}

The ellipses describe omitted display detail, not literal dataset content. The executable converter writes complete canonical JSON strings. It does not build a prompt by concatenating unescaped field values into a JSON fragment.

Our instruction representation uses instruction and response; completion uses prompt and completion. A narrow adapter maps both into the same two-message educational conversation before tokenisation. That is a deliberate equivalence in this implementation. A base language model receiving a literal completion prefix, or an instruction model receiving its native template, needs its own checked adapter. Calling three fields different names is not a compatibility result for those models.

Derive the identities that a conversion must preserve

Let $r$ be the canonical record, $a$ its annotation, $P$ the policy and $F$ a renderer version and format. The rendered example is

$$ e=R_F(r,a,P). $$
Its digest binds all four inputs through their identities and the resulting payload. A source revocation changes whether $e$ may be used even if its text remains the same. A label revision changes the annotation digest and invalidates dependent renderings. A renderer revision changes the representation identity even when a semantic incident is unchanged.

This gives a useful distinction between semantic and representational changes. Reindent a JSON file without changing its parsed object: the file-byte digest changes, while the project-defined canonical-object digest can remain equal. Change the label from low to high severity: the annotation identity and dependent examples must change. Change policy p1 to p2: the incident’s policy-independent semantic identity can remain equal, but its input, annotation and rendering identities change.

FineTuneLab’s deterministic JSON encoding sorts object keys, preserves array order, uses UTF-8 and rejects non-finite numbers. It is a project convention, not a claim to implement every requirement of a general canonicalisation standard. The JSON specification, RFC 8259, defines the interchange format; a reproducible experiment additionally needs its own decisions about ordering, duplicate keys and exact bytes.

An auditor should therefore retain both kinds of digest where relevant. A raw file digest tells us which file was ingested. A semantic or canonical digest tells us which explicitly normalised object was interpreted. Neither identifies the rightful owner or authenticates the producer.

Serialize records without changing their boundaries

JSONL places one complete JSON value on each line. A newline inside an incident is escaped within its JSON string; it must not create a second training record. Use a JSON serializer for every row, not string interpolation with a trailing newline. Reject duplicate object keys rather than accepting a parser-dependent winner.

Stable record ordering is another part of reproduction. Sorting by a declared record ID makes an export deterministic, but it does not randomise a training curriculum. Training order and sampler state are separate configuration choices. If records are sorted by class and consumed sequentially, that order can change learning even when the file contains the same set of examples.

The converter preserves source, record, semantic, annotation, task, policy and renderer identities in the envelope. The reader-facing answer remains the task output, not that envelope. Dropping provenance during export would make later deletion and contamination tracing harder even if it produced the same target tokens.

For a multi-turn conversation, role order is meaningful. An earlier assistant answer is part of the context of a later answer. A system message is a template role, not evidence that every instruction inside it is authoritative. A tool result is observations supplied by another component, not automatically a desired assistant response. Flattening all three into one string can change the learning objective and erase the boundaries needed to construct the loss mask.

Endings and truncation are semantic choices

The response must have an explicit ending convention. Our educational template adds EOS after each message and later supervises assistant content and its ending. It does not ask the stored JSON answer to contain a literal EOS control token. Rendering the template twice would duplicate control structure; putting a control-token spelling inside ordinary incident content must not manufacture a new role.

The separate byte tokenizer from Tokens, Chat Templates and Special Tokens makes that distinction inspectable. Ordinary content is encoded as bytes, while the compiler inserts control IDs. A real tokenizer with special-string handling requires its own tested injection boundary. This example does not establish that arbitrary model tokenizers share the byte compiler’s behaviour.

Truncation can erase the policy, remove the response, or leave half a structured object. A syntactically valid source row does not imply a valid training window. The educational compiler refuses silent clipping; an explicit whole-turn policy can retain a completed earlier response. The next chapter checks the exact targets surviving such a choice.

Packing several examples into one window adds another boundary: the second example may attend to the first unless attention is isolated. Merely inserting EOS does not prove independence. We initially keep one conversation per row and reject unsupported packing rather than silently presenting cross-example context as an optimisation with unchanged semantics.

Inspect the actual boundaries

The executable inspection compiles three canonical records: an access incident under p1 and p2, and an insufficient report under p1. It recovers their original input, policy and labels after each conversion. The measured unmerged-byte/template counts are:

Record Single-answer instruction, completion, chat or structured form: tokens / targets Repeated-answer multi-turn: tokens / targets
Access under p1 471 /114 637 /228
Same access under p2 471 /114 637 /228
Insufficient report under p1 479 /119 650 /238

The four single-answer forms are deliberately adapted into identical educational messages, so their counts agree. That is a checked local adapter property, not a claim that four native pretrained templates are interchangeable. The multi-turn form adds a user request to repeat the complete object under the same policy, then repeats the answer. Its doubled target count is intentional; the earlier answer is already context, so this form cannot support an independent prediction claim. Every ending is a compiler-inserted control ID rather than text concatenated onto an answer.

Run examples/part_iii/format_inspection.py in the CPU service with a fresh --output path to inspect the complete rendered strings, message endings and tokenizer identity. The rows are measured representations, not model outputs or evidence that these long records fit the earlier small model.

Reconstruct expected output when validating conversions

A submitted rendering can carry a correct hash for incorrect content. An attacker or faulty converter can replace the assistant answer with {} and recompute the digest. The useful check reconstructs the expected rendering from the trusted canonical record, annotation, policy and renderer, then compares the complete result.

The executable tests perform that attack and receive RENDER_CONTENT. They also revise annotation history without regenerating the dependent example, remove source provenance, select an unsupported format and pair a record with another annotation. The valid chat, instruction and completion forms produce equal educational messages and three distinct rendering digests.

These checks establish preservation within the implemented adapters. They do not establish that the label is true, that the task is representative, or that a downstream trainer actually honours intended_supervision. That field declares a purpose; it does not enforce it. Chapter 16 will compile the declaration into exact target positions and compare the resulting gradients.

The offline phase-A command from the previous chapter exports the policy pair and all three representations. Inspect the complete user content, assistant content and outer provenance separately. In particular, confirm that category and severity appear in the answer rather than being accidentally supplied through annotation metadata in the prompt. A correct conversion is an information-boundary claim before it is a formatting convenience.

An identifier can disclose the answer

The original pilot rendered family-derived incident IDs such as security_active-0-p1. An identifier-only keyword rule recovered all 36 categories on which it made a prediction. That is a shortcut, even when the incident prose also supports the answer: it allows a model to succeed through forbidden metadata. Correct JSON and correct labels do not repair the input.

Revision 0.2 separates internal family/record provenance from the model-visible identifier. The latter is i- followed by 24 hexadecimal characters from a domain-separated hash of text, information status and reported impact. Category, severity, route, family, split and policy version are excluded from construction. A p1/p2 incident pair retains the same visible ID. Only the five declared incident input fields and the active policy enter the rendered prompt; target JSON and internal debug fields are rejected.

Across the 80-record candidate, the identifier keyword rule covers zero records and therefore makes no successful predictions. The majority-class baseline is 16/80 = 0.20. Abstention by the keyword rule is not 80 correct classifications. Construction invariance and explicit metadata mutation tests provide additional evidence, but neither hash appearance nor zero keyword coverage proves statistical independence for an unseen population. Legitimate category evidence remains in the incident text.

Further Reading

  1. Jason Wei et al., “Finetuned Language Models Are Zero-Shot Learners” (2021). Read the source. Converts dozens of datasets into instruction templates with cluster holdouts, showing what a format conversion must preserve.
  2. Yizhong Wang et al., “Self-Instruct” (2022). Read the source. Bootstraps instructions from model outputs with filtering, illustrating the provenance and circularity risks the chapter guards against with renderer identities.
  3. Rohan Taori et al., “Alpaca: A Strong, Replicable Instruction-Following Model” (2023). Read the source. Provides a low-cost 52K-demonstration SFT recipe whose prompt structure clarifies the instruction/response boundary.