Why it matters
A fine-tune almost always improves the metric it was trained toward, because that is what optimisation does. The decision needs more: whether the gain survives on unseen prompts, whether general behaviour regressed, and whether the improvement justifies a second model in production. Without a paired design and a regression slice, teams ship something that wins the benchmark and loses the product.
Mental model
Evaluating a fine-tune is a drug trial, not a victory lap: you need a control group and a list of side effects. The analogy breaks because you can always run both models on the same prompt, so the comparison is paired and far cheaper than a trial.
How it works
- 1Write the decision the evaluation must support, such as ship, iterate or abandon, and the threshold for each.
- 2Build a held-out set from prompts, grouped so paraphrases of a training prompt cannot appear.
- 3Score the tuned model and the base model on the same items, and report the paired difference with its uncertainty.
- 4Add regression slices: general instructions, refusals, long inputs and the previous task mix.
- 5Measure operational properties: latency, tokens per response and cost per request at the serving configuration.
- 6Inspect the failures the aggregate hides, stratified by category and by input length.
Step 1 of 6
Write the decision the evaluation must support, such as ship, iterate or abandon, and the threshold for each.
Figure. The mechanism in this page's own steps. The figure adds sequence, not new claims.
What to notice. A step can be read alone, but the order is the mechanism. The control lets you stop on any one.
Text version
The page's steps: 1. Write the decision the evaluation must support, such as ship, iterate or abandon, and the… 2. Build a held-out set from prompts, grouped so paraphrases of a training prompt cannot app… 3. Score the tuned model and the base model on the same items, and report the paired differe… 4. Add regression slices: general instructions, refusals, long inputs and the previous task … 5. Measure operational properties: latency, tokens per response and cost per request at the … 6. Inspect the failures the aggregate hides, stratified by category and by input length.
- Write the decision the evaluation must support, such as ship, iterate or abandon, and the threshold for each.
- Build a held-out set from prompts, grouped so paraphrases of a training prompt cannot appear.
- Score the tuned model and the base model on the same items, and report the paired difference with its uncertainty.
- Add regression slices: general instructions, refusals, long inputs and the previous task mix.
- Measure operational properties: latency, tokens per response and cost per request at the serving configuration.
- Inspect the failures the aggregate hides, stratified by category and by input length.
Trade-offs
| Choice | Good for | Cost | Fails when |
|---|---|---|---|
| Exact match or rubric scoring | Cheap and repeatable | Misses valid paraphrases | Output is open-ended |
| LLM judge | Scales to open-ended tasks | Judge bias and instability | The judge shares the tuned model's style |
| Human review | Trustworthy on nuance | Slow and expensive | You need per-commit signal |
| Small targeted set | Fast iteration | Wide confidence intervals | You need a ship decision |
Good for
Cheap and repeatable
Cost
Misses valid paraphrases
Fails when
Output is open-ended
- Output is open-ended
- The judge shares the tuned model's style
- You need per-commit signal
- You need a ship decision
Figure. Each choice with what it buys, what it costs, and the case where it loses.
What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.
Text version
Four trade-offs: Exact match or rubric scoring is good for Cheap and repeatable, costs Misses valid paraphrases, and fails when Output is open-ended. LLM judge is good for Scales to open-ended tasks, costs Judge bias and instability, and fails when The judge shares the tuned model's style. Human review is good for Trustworthy on nuance, costs Slow and expensive, and fails when You need per-commit signal. Small targeted set is good for Fast iteration, costs Wide confidence intervals, and fails when You need a ship decision
Failure modes
- Prompt leakage. Tuned scores are high and production disagrees. Debug: nearest-neighbour each held-out prompt against training prompts and inspect the closest matches.
- Unpaired comparison. The two models were scored on different samples and the difference is sampling noise. Debug: re-score both on an identical item list and compute the paired difference.
- Metric gaming. Length or formatting drives the score up while quality is flat. Debug: correlate score with response length and re-score length-normalised outputs.
In production
Keep the evaluation set versioned and frozen, and re-run it for every checkpoint you consider shipping. Monitor the same slices online so offline and live measurements stay comparable.
Common misconceptions
People think: A rising eval score means the fine-tune worked.
Actually: It means the model improved on that eval; regression slices and cost decide whether it ships.
People think: Training and validation loss are enough.
Actually: They measure fit to your data, not behaviour on the prompts users send.
Interview angle
Structure the answer around three questions and a paired design, and mention the regression slice that most teams forget. A strong candidate names the decision threshold before seeing results. Weak answers quote a benchmark delta with no held-out protocol and no operational measurement.
Sources
- Liang et al., Holistic Evaluation of Language Models — multi-metric evaluation and the case against single scores.
- Zheng et al., Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena — biases that make judge scores unreliable.
- Hugging Face, Evaluate documentation — measurement implementations and comparison patterns.