Why it matters
Prompt changes are usually judged by reading a few outputs, and a convincing sample is easy to produce. That process cannot distinguish a real improvement from a lucky draw, and it cannot catch regressions on cases nobody thought to try. Testing replaces judgement-by-example with a repeatable measurement, which is what makes iteration fast rather than risky. It also gives reviewers something to argue about other than taste.
Mental model
Prompt testing is a driving test with a fixed route: everyone is measured on the same manoeuvres. The metaphor breaks because the route can be memorised, and a prompt tuned to the test set may fail on the road. The examiner also never sees the traffic you meet later.
How it works
Scoring window · illustrative tokens
700 input+ 300 reserve= 1,000 of 1,800
800 free
Step 1Exact match
A label comparison is code, not a call: 700 tokens of case and expected label, 800 free, and no scoring window to pay for. It only works when the answer can be compared exactly.
Output reserve
300No judge call: the 800 free tokens are the cost the exact-match check does not spend.
Pick a section to read its token count, its role and its state.
Figure. Model-as-judge scoring is a second call per case: a rubric, the case input, the candidate answer and sometimes a reference share one judge window. Rubric scoring fits with 40 tokens free; adding a reference cuts 140 tokens from the answer being graded.
What to notice. Counts are authored for a 1,800-token judge window with a 300-token verdict reserve; no judge was called and no suite was run. The judge window is a composition the page names as a cost, not a number any provider publishes. Exact match is drawn as a comparison in code with no window; human review is omitted.
Text version
A 1,800-token judge window with a 300-token verdict reserve, so 1,500 tokens are available. The default first step is exact match: the case input 640 and the expected label 60, for 700 tokens and 800 free, with no scoring call at all. The second step is a rubric judge: judge instructions 90, rubric 320, case input 640 and candidate answer 410, for 1,460 tokens and 40 free. The third step adds a 180-token reference answer and runs 140 over the window, with the cells crossing the cap rule into the reserve. The fourth step keeps the reference and cuts the candidate answer from 410 to 270 tokens, filling the window exactly — the judge now grades a truncated answer. Selecting a section shows its count and role.
- Collect cases from real traffic, including failures, edge cases and the common path, and label the expected output or property.
- Choose a scoring rule per case type: exact match for labels, field-level checks for extraction, a rubric for prose.
- Freeze a development split for iteration and a held-out split used only for acceptance.
- Run every candidate prompt on both splits and record scores with the prompt and model version.
- Set an acceptance threshold before looking at results, including a tolerance for noise.
- Refresh the set on a schedule with newly observed cases, and retire stale ones.
Trade-offs
| Choice | Good for | Cost | Fails when |
|---|---|---|---|
| Exact match | Objective and cheap | Punishes valid variation | The output is free-form |
| Model-as-judge | Scales to open-ended output | Judge bias and cost | The judge is the same model being tested |
| Human review | Best judgement | Slow and expensive | You need per-commit feedback |
| Large synthetic set | Broad coverage | May not resemble real traffic | The distribution is unusual |
Good for
Objective and cheap
Cost
Punishes valid variation
Fails when
The output is free-form
- The output is free-form
- The judge is the same model being tested
- You need per-commit feedback
- The distribution is unusual
Figure. Each choice with what it buys, what it costs, and the case where it loses.
What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.
Text version
Four trade-offs: Exact match is good for Objective and cheap, costs Punishes valid variation, and fails when The output is free-form. Model-as-judge is good for Scales to open-ended output, costs Judge bias and cost, and fails when The judge is the same model being tested. Human review is good for Best judgement, costs Slow and expensive, and fails when You need per-commit feedback. Large synthetic set is good for Broad coverage, costs May not resemble real traffic, and fails when The distribution is unusual
Failure modes
- Overfitting to the test set. Scores rise while production quality falls. Debug: compare development and held-out scores, and refresh with recent traffic.
- Scoring the wrong thing. The metric rewards format and ignores correctness. Debug: inspect a sample of high-scoring failures by hand.
- Noisy comparison. A one-case difference is treated as an improvement. Debug: report confidence intervals or run the set multiple times with sampling.
In production
Run the suite on every prompt change and on every model upgrade, and keep results with the version ids. Sample production outputs into the set so it tracks the traffic.
Common misconceptions
People think: A prompt that works on ten examples is ready.
Actually: Ten examples show plausibility; acceptance needs a set sized for the regressions you care about.
People think: Model-as-judge removes the need for labels.
Actually: A judge needs a reference or rubric, and it inherits biases that must be measured against human labels.
Interview angle
Describe the set, the scoring rule and the split, then explain how the result changes a decision. Strong answers mention noise, overfitting and refresh. Weak answers say they try a few prompts and compare the outputs by eye.
Sources
- Zheng et al., Judging LLM-as-a-Judge — automated judging, agreement and known biases.
- Liang et al., Holistic Evaluation of Language Models — structured evaluation across scenarios and metrics.
- OpenAI, Evals repository — an open framework for repeatable model evaluation.