Skip to content
AI Engineer Guide.

concept · concepts · intermediate · concept 3 of 22

Instruction tuning

0 of 22 complete

Why it matters

A pretrained model is a text continuer. Given a question it may answer, restate the question or produce a list, because nothing in pretraining specified that a request should be obeyed. Instruction tuning supplies that contract with many pairs of a natural-language request and the response a careful person would give. The engineering problem is coverage, since the model generalises across descriptions and fails on shapes it never saw.

Mental model

It is like teaching a new employee the house style by showing worked examples from several departments rather than training them for one job. The analogy breaks because the model does not infer intent; it pattern-matches phrasing, so an instruction written in an unfamiliar register can still slip through.

How it works

Four format layers, one supervised layerInteractive
signalbypass

Step 1 of 6

Hold templates out

The unit of evaluation is a whole template, not a random example. Splitting examples lets near-identical wording sit on both sides and inflates the score.

Active layers

  • Task description

Hover a layer to read what it holds.

Next: Write the format layers

Supervised
response tokens
Held out
whole templates
Adds
format, not facts
Step 1/6 · Hold templates out

Figure. The format layers of one instruction example, in template order, and the single layer the loss selects. The system preamble, instruction and optional input are prompt; only the response is supervised. The dashed lane is the supervision signal, not a runtime path. Category balance and template diversity are named in the steps rather than drawn.

What to notice. The vertical order is the template's order, not the order the model computes: the response is assembled after the prompt but the forward pass runs once over the whole sequence. The mask is what turns instructions into context rather than targets. The figure does not show evaluation; the last step names held-out templates as the test, because a category win can hide a failure elsewhere.

Text version

A vertical stack. A task description enters a group for one example, which holds system preamble, instruction, optional input and response in template order. The assembled example feeds a chat template, then a forward pass, then a loss. A dashed lane carries the response directly to the loss labelled supervised; the loss feeds an update and an instruction follower. Six steps cover holding templates out, the prompt layers, the response, assembly and forward, the masked loss, and evaluation.

  1. Collect tasks across the categories you care about, including extraction, classification, rewriting, refusal and multi-step reasoning.
  2. Hold out whole task templates rather than random examples, so evaluation measures generalisation to unseen instructions.
  3. Write responses in the exact register and length you want at inference, since the model copies style more reliably than it follows a description of style.
  4. Balance categories by the count of distinct templates, or the largest category will dominate the loss.
  5. Compute loss on response tokens only and mask the instruction, so the model learns to answer rather than to reproduce requests.
  6. Evaluate on held-out templates and on a sample of the base model's traffic to catch regressions outside the target distribution.

Trade-offs

ChoiceGood forCostFails when
Broad task mixGeneralises to unseen instructionsAnnotation cost across many categoriesYou need one narrow, high-volume behaviour
Narrow task mixHigh accuracy on the target taskPoor transfer and brittle phrasingUsers phrase requests in many ways
Model-written responsesCheap and scalableInherits the writer model's errorsCorrectness matters more than style
Human-written responsesFaithful style and judgementSlow and expensiveVolume is the binding constraint
Trade-off mapInteractive

Good for

Generalises to unseen instructions

Cost

Annotation cost across many categories

Fails when

You need one narrow, high-volume behaviour

  1. You need one narrow, high-volume behaviour
  2. Users phrase requests in many ways
  3. Correctness matters more than style
  4. Volume is the binding constraint

Figure. Each choice with what it buys, what it costs, and the case where it loses.

What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.

Text version

Four trade-offs: Broad task mix is good for Generalises to unseen instructions, costs Annotation cost across many categories, and fails when You need one narrow, high-volume behaviour. Narrow task mix is good for High accuracy on the target task, costs Poor transfer and brittle phrasing, and fails when Users phrase requests in many ways. Model-written responses is good for Cheap and scalable, costs Inherits the writer model's errors, and fails when Correctness matters more than style. Human-written responses is good for Faithful style and judgement, costs Slow and expensive, and fails when Volume is the binding constraint

Failure modes

  • Template leakage. Held-out examples reuse the wording of training instructions and scores look excellent. Debug: cluster instruction texts and check that no evaluation cluster appears in the training set.
  • Answer-only training. The model learns to emit a plausible answer to any input, including empty or hostile ones. Debug: include refusal and clarification examples and score them as their own category.
  • Style override. Responses are correct but verbose because the training answers were verbose. Debug: measure mean response length against the target and inspect the longest deciles.

In production

Version the task mix and re-run the held-out templates whenever it changes. Track refusal rate and response length on live traffic, since both drift before accuracy does.

Common misconceptions

People think: Instruction tuning adds knowledge.

Actually: It shapes how the model responds to requests; facts still come from pretraining or from context you supply.

People think: More examples per task is the lever.

Actually: Distinct task templates generalise; repeating one template mostly teaches that template.

Interview angle

The generalisation claim is the heart of it: the model learns to map descriptions to behaviour, so evaluation must hold out templates. Mention loss masking and category balance. Candidates who describe instruction tuning as teaching facts, or who evaluate with a random split of one template, have missed what the method does.

Sources