Skip to content
AI Engineer Guide.

concept · evaluation · intermediate · concept 27 of 28

Eval-driven development

0 of 28 complete

Why it matters

Prompt and retrieval changes are hard to reason about, so teams ship them on intuition and discover regressions later. Writing the case first turns the intended behaviour into a specification that fails before the change and passes after. The practice borrows its discipline from test-driven development, with one difference: the oracle is noisy, so a single run is not a verdict and thresholds need margins.

Mental model

It is TDD for probabilistic behaviour. The metaphor breaks because the test is stochastic, so green is a distribution rather than a fact, and a single run can pass for reasons that have nothing to do with the change under test.

How it works

Mechanism, step by stepInteractive
  1. 1Take one real failure and write it as a case with an input, a check and a note on the harm.
  2. 2Confirm the case fails on the current system for the stated reason.
  3. 3Make the smallest prompt, retrieval or tool change that addresses it.
  4. 4Run the full suite to catch regressions, repeating runs where sampling is stochastic.
  5. 5Keep the change only if the target case passes and no slice regresses.
  6. 6Commit the case with the change so the specification grows with the product.

Step 1 of 6

Take one real failure and write it as a case with an input, a check and a note on the harm.

Step 1/6 · step 1

Figure. The mechanism in this page's own steps. The figure adds sequence, not new claims.

What to notice. A step can be read alone, but the order is the mechanism. The control lets you stop on any one.

Text version

The page's steps: 1. Take one real failure and write it as a case with an input, a check and a note on the har… 2. Confirm the case fails on the current system for the stated reason. 3. Make the smallest prompt, retrieval or tool change that addresses it. 4. Run the full suite to catch regressions, repeating runs where sampling is stochastic. 5. Keep the change only if the target case passes and no slice regresses. 6. Commit the case with the change so the specification grows with the product.

Green is a rate, not a runInteractive

The interactive figure loads with JavaScript. The text version below describes the same frame.

Figure. The same 32 eval cases bucketed by how often they pass over repeated runs. One run makes every case look binary; ten and twenty reruns move a block into the middle, where the case is neither a specification that holds nor one that fails. Counts are illustrative.

What to notice. One run reports 26 passes and six failures. Over twenty reruns the six failures stay failures, 23 cases settle above 80%, and six cases sit between 20 and 80%: they pass often enough to look green and fail often enough that a single run decides which answer you see. The target case and its verdict need a pass rate before either a keep or a revert is honest.

Text version

Horizontal bars for five pass-rate bands, counting eval cases. After twenty reruns, 23 cases pass in at least 80% of runs, two sit in 60-80%, two in 40-60%, two in 20-40% and three in 0-20%. A two-button control switches to ten reruns, where the counts are 22, 2, 3, 2 and 3, and to one run, where all 32 cases fall in the top band or the bottom band and the middle three are empty. A panel reports cases above 80%, cases between 20 and 80% and whether the verdict is a rate.

  1. Take one real failure and write it as a case with an input, a check and a note on the harm.
  2. Confirm the case fails on the current system for the stated reason.
  3. Make the smallest prompt, retrieval or tool change that addresses it.
  4. Run the full suite to catch regressions, repeating runs where sampling is stochastic.
  5. Keep the change only if the target case passes and no slice regresses.
  6. Commit the case with the change so the specification grows with the product.

Trade-offs

ChoiceGood forCostFails when
Case per bugDirect and groundedSuite grows without structureVolume of failures is high
Large suite before buildingDesign pressureSlow start, speculative casesProduct direction is unclear
Strict no-regression gateStable qualityBlocks deliberate trade-offsTrade-off is intentional
Manual spot checksFastNo memory, no comparabilityTeam grows
Trade-off mapInteractive

Good for

Direct and grounded

Cost

Suite grows without structure

Fails when

Volume of failures is high

  1. Volume of failures is high
  2. Product direction is unclear
  3. Trade-off is intentional
  4. Team grows

Figure. Each choice with what it buys, what it costs, and the case where it loses.

What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.

Text version

Four trade-offs: Case per bug is good for Direct and grounded, costs Suite grows without structure, and fails when Volume of failures is high. Large suite before building is good for Design pressure, costs Slow start, speculative cases, and fails when Product direction is unclear. Strict no-regression gate is good for Stable quality, costs Blocks deliberate trade-offs, and fails when Trade-off is intentional. Manual spot checks is good for Fast, costs No memory, no comparability, and fails when Team grows

Failure modes

  • Case written to pass. The expected output is copied from current behaviour including its bug. Debug: check the case fails on the commit before the fix.
  • Flaky verdict. A stochastic case flips between runs and blocks unrelated changes. Debug: run the case repeatedly and measure its pass distribution before gating on it.
  • Suite rot. Cases reference prompts or endpoints that no longer exist. Debug: run the suite on main weekly and fail on errors, not only on score drops.

In production

Run the suite on every change to prompts, retrieval or models, and store per-case results. Track suite runtime and prune cases that no longer discriminate.

Common misconceptions

People think: Evals are a QA phase after building.

Actually: They are the specification, and writing them afterwards usually encodes whatever the system already does.

People think: A green suite means the feature is good.

Actually: It means the cases you wrote pass, and coverage of the behaviour is the open question.

Interview angle

Start from a real failing case, check that it fails for the right reason and keep the case with the fix. A strong answer covers stochastic margins and suite pruning. Writing evals after the feature and using them as a dashboard, rather than a gate, is the weak response.

Sources