Skip to content
AI Engineer Guide.

concept · concepts · intermediate · concept 17 of 22

Fine-tuning evaluation

0 of 22 complete

Why it matters

A fine-tune almost always improves the metric it was trained toward, because that is what optimisation does. The decision needs more: whether the gain survives on unseen prompts, whether general behaviour regressed, and whether the improvement justifies a second model in production. Without a paired design and a regression slice, teams ship something that wins the benchmark and loses the product.

Mental model

Evaluating a fine-tune is a drug trial, not a victory lap: you need a control group and a list of side effects. The analogy breaks because you can always run both models on the same prompt, so the comparison is paired and far cheaper than a trial.

How it works

Mechanism, step by stepInteractive
  1. 1Write the decision the evaluation must support, such as ship, iterate or abandon, and the threshold for each.
  2. 2Build a held-out set from prompts, grouped so paraphrases of a training prompt cannot appear.
  3. 3Score the tuned model and the base model on the same items, and report the paired difference with its uncertainty.
  4. 4Add regression slices: general instructions, refusals, long inputs and the previous task mix.
  5. 5Measure operational properties: latency, tokens per response and cost per request at the serving configuration.
  6. 6Inspect the failures the aggregate hides, stratified by category and by input length.

Step 1 of 6

Write the decision the evaluation must support, such as ship, iterate or abandon, and the threshold for each.

Step 1/6 · step 1

Figure. The mechanism in this page's own steps. The figure adds sequence, not new claims.

What to notice. A step can be read alone, but the order is the mechanism. The control lets you stop on any one.

Text version

The page's steps: 1. Write the decision the evaluation must support, such as ship, iterate or abandon, and the… 2. Build a held-out set from prompts, grouped so paraphrases of a training prompt cannot app… 3. Score the tuned model and the base model on the same items, and report the paired differe… 4. Add regression slices: general instructions, refusals, long inputs and the previous task … 5. Measure operational properties: latency, tokens per response and cost per request at the … 6. Inspect the failures the aggregate hides, stratified by category and by input length.

  1. Write the decision the evaluation must support, such as ship, iterate or abandon, and the threshold for each.
  2. Build a held-out set from prompts, grouped so paraphrases of a training prompt cannot appear.
  3. Score the tuned model and the base model on the same items, and report the paired difference with its uncertainty.
  4. Add regression slices: general instructions, refusals, long inputs and the previous task mix.
  5. Measure operational properties: latency, tokens per response and cost per request at the serving configuration.
  6. Inspect the failures the aggregate hides, stratified by category and by input length.

Trade-offs

ChoiceGood forCostFails when
Exact match or rubric scoringCheap and repeatableMisses valid paraphrasesOutput is open-ended
LLM judgeScales to open-ended tasksJudge bias and instabilityThe judge shares the tuned model's style
Human reviewTrustworthy on nuanceSlow and expensiveYou need per-commit signal
Small targeted setFast iterationWide confidence intervalsYou need a ship decision
Trade-off mapInteractive

Good for

Cheap and repeatable

Cost

Misses valid paraphrases

Fails when

Output is open-ended

  1. Output is open-ended
  2. The judge shares the tuned model's style
  3. You need per-commit signal
  4. You need a ship decision

Figure. Each choice with what it buys, what it costs, and the case where it loses.

What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.

Text version

Four trade-offs: Exact match or rubric scoring is good for Cheap and repeatable, costs Misses valid paraphrases, and fails when Output is open-ended. LLM judge is good for Scales to open-ended tasks, costs Judge bias and instability, and fails when The judge shares the tuned model's style. Human review is good for Trustworthy on nuance, costs Slow and expensive, and fails when You need per-commit signal. Small targeted set is good for Fast iteration, costs Wide confidence intervals, and fails when You need a ship decision

Failure modes

  • Prompt leakage. Tuned scores are high and production disagrees. Debug: nearest-neighbour each held-out prompt against training prompts and inspect the closest matches.
  • Unpaired comparison. The two models were scored on different samples and the difference is sampling noise. Debug: re-score both on an identical item list and compute the paired difference.
  • Metric gaming. Length or formatting drives the score up while quality is flat. Debug: correlate score with response length and re-score length-normalised outputs.

In production

Keep the evaluation set versioned and frozen, and re-run it for every checkpoint you consider shipping. Monitor the same slices online so offline and live measurements stay comparable.

Common misconceptions

People think: A rising eval score means the fine-tune worked.

Actually: It means the model improved on that eval; regression slices and cost decide whether it ships.

People think: Training and validation loss are enough.

Actually: They measure fit to your data, not behaviour on the prompts users send.

Interview angle

Structure the answer around three questions and a paired design, and mention the regression slice that most teams forget. A strong candidate names the decision threshold before seeing results. Weak answers quote a benchmark delta with no held-out protocol and no operational measurement.

Sources