Why it matters
Ordinary software's contract is a function: same input, same output. A model-backed system breaks that assumption, and with it the testing, debugging and rollback practices built on it. The response is not to pretend determinism but to define behaviour statistically: what fraction of cases must pass, which failures are unacceptable, and how much variance the product can absorb.
Mental model
Probabilistic software is a manufacturing process with tolerances, not a calculator. The metaphor breaks because a factory measures every unit against a spec, while here the spec is a rubric and measuring each unit costs another model call or a human.
How it works
- 1Write the behaviour as a specification with a tolerance: a pass rate on a labelled set, not an expected string.
- 2Fix what you can (seed, temperature, decoding constraints, pinned model version) and record it with the output.
- 3Sample: run the evaluation several times and report the spread, not the best run.
- 4Separate unacceptable failure classes from tolerable variance and treat them differently.
- 5Add deterministic checks around the model for anything that must never happen.
- 6Track production distributions, because a shift in inputs changes the distribution of outputs.
Step 1 of 6
Write the behaviour as a specification with a tolerance: a pass rate on a labelled set, not an expected string.
Figure. The mechanism in this page's own steps. The figure adds sequence, not new claims.
What to notice. A step can be read alone, but the order is the mechanism. The control lets you stop on any one.
Text version
The page's steps: 1. Write the behaviour as a specification with a tolerance: a pass rate on a labelled set, n… 2. Fix what you can (seed, temperature, decoding constraints, pinned model version) and reco… 3. Sample: run the evaluation several times and report the spread, not the best run. 4. Separate unacceptable failure classes from tolerable variance and treat them differently. 5. Add deterministic checks around the model for anything that must never happen. 6. Track production distributions, because a shift in inputs changes the distribution of out…
- Write the behaviour as a specification with a tolerance: a pass rate on a labelled set, not an expected string.
- Fix what you can (seed, temperature, decoding constraints, pinned model version) and record it with the output.
- Sample: run the evaluation several times and report the spread, not the best run.
- Separate unacceptable failure classes from tolerable variance and treat them differently.
- Add deterministic checks around the model for anything that must never happen.
- Track production distributions, because a shift in inputs changes the distribution of outputs.
Trade-offs
| Choice | Good for | Cost | Fails when |
|---|---|---|---|
| Greedy decoding | More repeatable outputs | Less variety and possible degeneration | The task benefits from diverse candidates |
| Sampling with a seed | Reproducible for debugging | Seed support varies and does not fix upstream non-determinism | You need bit-identical reruns across hardware |
| Statistical acceptance | Matches the real contract | Needs many trials and a labelled set | Traffic is small or labels are expensive |
| Strict deterministic wrapper | Bounds the damage | Limits useful output | The task is genuinely open-ended |
Good for
More repeatable outputs
Cost
Less variety and possible degeneration
Fails when
The task benefits from diverse candidates
- The task benefits from diverse candidates
- You need bit-identical reruns across hardware
- Traffic is small or labels are expensive
- The task is genuinely open-ended
Figure. Each choice with what it buys, what it costs, and the case where it loses.
What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.
Text version
Four trade-offs: Greedy decoding is good for More repeatable outputs, costs Less variety and possible degeneration, and fails when The task benefits from diverse candidates. Sampling with a seed is good for Reproducible for debugging, costs Seed support varies and does not fix upstream non-determinism, and fails when You need bit-identical reruns across hardware. Statistical acceptance is good for Matches the real contract, costs Needs many trials and a labelled set, and fails when Traffic is small or labels are expensive. Strict deterministic wrapper is good for Bounds the damage, costs Limits useful output, and fails when The task is genuinely open-ended
Failure modes
- Single-run test. A flaky behaviour passes once and fails in production. Debug: run the same case ten times and inspect the pass distribution before merging.
- Hidden non-determinism. Outputs differ even at temperature zero because of batching or version changes. Debug: pin versions and compare repeated calls with identical inputs.
- Distribution shift. Quality falls without a code change because inputs changed. Debug: compare input distributions week over week against the evaluation set.
In production
Monitor output distributions and sampled quality, not just error rates. Keep an error budget for quality and alert when the sampled pass rate falls below it.
Common misconceptions
People think: Temperature zero makes the system deterministic.
Actually: It reduces sampling variance; batching, hardware and model updates can still change outputs.
People think: Probabilistic means untestable.
Actually: It means tests assert distributions and properties rather than exact strings.
Interview angle
Strong answers replace exact assertions with properties, pass rates and tolerance bands, and describe what is pinned to make a failure reproducible. Weak answers claim the system is deterministic because temperature is zero.
Sources
- Sculley et al., Hidden Technical Debt in Machine Learning Systems — why data-dependent behaviour erodes ordinary testing assumptions.
- Karpathy, Software 2.0 — behaviour specified by data rather than by explicit instructions.
- Breck et al., The ML Test Score — testing practices for systems whose outputs vary.