Why it matters
A model that returns a single answer hides its uncertainty. A model that returns a distribution exposes it, and then you have to decide what to do with it: take the most likely token, sample from the tail, reject the request, or escalate to a human.
That decision is where most production incidents live. Greedy decoding on a task that needs diversity produces repetition; high-temperature sampling on a task that needs exact extraction produces plausible nonsense; an uncalibrated confidence score used as a routing threshold sends the wrong queries to the expensive path. None of these are model bugs. They are misuse of probabilities.
Probability also underpins evaluation. "The new prompt won 7 of 10 cases" is not evidence of improvement. Understanding variance and sample size is what stops a team from shipping noise.
Mental model: a budget split across outcomes
Think of a probability distribution as a fixed budget of 1.0 that must be divided among all possible outcomes. A fair coin splits it 0.5 and 0.5. A model's next-token distribution splits it across tens of thousands of tokens, with most of the budget concentrated on a few plausible continuations.
Conditioning is the same budget re-divided after new information arrives: given that the document is about billing, the budget shifts away from travel tokens.
The analogy breaks in a specific way. The budget is not a property of the world; it is a property of the model's beliefs. Two models can assign different budgets to the same event, and both can be wrong. Probability from a model is a claim about its own uncertainty, not a measurement of reality.
How it works
Ordered by logit, descending. The order is fixed at every temperature, so the bars never swap places.
T = 0.5: A 0.862 · B 0.117 · C 0.019 · D 0.002 — total 1.000.
- Top share
- 0.862
- Bottom share
- 0.002
- Printed total
- 1.000
T = 0.5 divides the logits before the softmax and sharpens it: outcome A rises to 0.862 and D falls to 0.002. No ranking changed; the budget was redistributed.
Values
| category | Probability |
|---|---|
| Outcome A · logit 2.0 | 0.862 |
| Outcome B · logit 1.0 | 0.117 |
| Outcome C · logit 0.1 | 0.019 |
| Outcome D · logit -1.0 | 0.002 |
Reference rule: even split of 4 · 0.250 at 0.250.
The logits are the page's own worked example and every bar is the exact softmax at the stated temperature, rounded by largest remainder so the printed total is 1.000. Four outcomes are shown rather than a vocabulary, and conditioning, calibration and Bayes' rule are not in the figure.
Figure. The page's own worked example — logits 2.0, 1.0, 0.1 and −1.0 — turned into probabilities at T = 0.5, 1 and 2. The dashed rule is 0.25, an even split of the budget across four outcomes. Every bar is the exact softmax of those logits at the stated temperature; the page prints the same three rows.
What to notice. The four shares always sum to 1.000; that is the whole claim of a distribution. Temperature moves mass between outcomes without changing which is most likely. The uniform rule is the comparison that matters: at T = 2 outcome D is still below an even share, and at T = 0.5 it is about 400 times smaller than A. This is a distribution over four outcomes, not a vocabulary, and it is not a calibrated probability that any outcome is correct.
Text version
A horizontal bar chart of four outcomes labelled A with logit 2.0, B with 1.0, C with 0.1 and D with minus 1.0, showing each probability. A dashed rule at 0.25 marks an even split across four outcomes. Three buttons set the temperature to 0.5, 1 or 2; the default is 1, where the shares are 0.638, 0.235, 0.095 and 0.032 and sum to 1.000. At 0.5 they run 0.862, 0.117, 0.019 and 0.002. At 2 they flatten to 0.451, 0.274, 0.174 and 0.101. The card reports the top share, bottom share and printed total for the current frame. Steps sharpen, flatten, then return to T = 1 and highlight A, D and the whole chart in turn.
- Define outcomes. The sample space is the set of things that can happen: the next token, the document's class, whether a user churns. Events are subsets of that space.
- Assign and normalise. Each outcome gets a non-negative number, and all numbers sum to 1. A model produces unnormalised scores (logits); softmax converts them into a valid distribution.
- Condition. is the probability of given that happened. Conditioning restricts the sample space and renormalises.
- Check independence before multiplying. only when and are independent. Retrieval scores, judge votes and repeated model calls are usually correlated, so naive multiplication overstates confidence.
- Use random variables to summarise. A random variable maps outcomes to numbers — the number of retries, the latency, the score. Its expectation is the long-run average and its variance is the spread.
- Pick a distribution that fits the outcome type. Bernoulli for a binary event, categorical for one of many tokens, Gaussian for continuous noise around a mean.
- Update with Bayes' rule. Start from a prior, multiply by the likelihood of the evidence, normalise. This is the formal version of "how much should this new signal change my mind".
- Work in log space. Products of many small probabilities underflow to zero in float32; sums of log-probabilities do not.
The math
Bayes' rule, in the form used for updating a belief:
Expectation and variance for a discrete random variable:
Softmax with a temperature turns logits into a distribution; sharpens it, flattens it:
A minimal implementation
import numpy as np
def softmax(logits: np.ndarray, temperature: float = 1.0) -> np.ndarray:
z = np.asarray(logits, dtype=np.float64) / temperature
z = z - z.max() # log-sum-exp shift: same result, no overflow
e = np.exp(z)
return e / e.sum()
def report_confidence(probs: np.ndarray) -> dict:
top = int(np.argmax(probs))
return {"token": top, "p": float(probs[top]),
"entropy": float(-(probs * np.log(probs + 1e-12)).sum())}
logits = np.array([2.0, 1.0, 0.1, -1.0])
for t in (0.5, 1.0, 2.0):
p = softmax(logits, temperature=t)
print(t, np.round(p, 3), report_confidence(p)["p"])
rng = np.random.default_rng(0)
draws = rng.choice(len(logits), size=10_000, p=softmax(logits))
print(np.bincount(draws, minlength=4) / 10_000) # converges to the distributionTwo details are load-bearing. The z.max() subtraction prevents exp overflow without changing the result. And p=softmax(logits) is only correct as a sampling distribution — reading p as "the probability the answer is right" is a different claim that needs calibration evidence.
Trade-offs
| Choice | Good for | Cost | Fails when |
|---|---|---|---|
| Greedy (argmax) | Extraction, classification, deterministic tests | No diversity, can loop | The task accepts several valid answers and variety matters |
| Temperature sampling | Brainstorming, paraphrasing | Unstable output, needs seeds | Output feeds a parser or a customer-facing fact |
| Top-p / top-k | Trimming the implausible tail | Can cut a valid rare token | The distribution is genuinely flat and the tail holds the answer |
| Raw model probability as confidence | Cheap triage signal | Often overconfident | Used as a user-facing risk score without calibration |
| Temperature scaling (post-hoc) | Calibrating an existing model | Needs held-out labelled data | Traffic distribution shifts after fitting |
Good for
Extraction, classification, deterministic tests
Cost
No diversity, can loop
Fails when
The task accepts several valid answers and variety matters
- The task accepts several valid answers and variety matters
- Output feeds a parser or a customer-facing fact
- The distribution is genuinely flat and the tail holds the answer
- Used as a user-facing risk score without calibration
- Traffic distribution shifts after fitting
Figure. Each choice with what it buys, what it costs, and the case where it loses.
What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.
Text version
Four trade-offs: Greedy (argmax) is good for Extraction, classification, deterministic tests, costs No diversity, can loop, and fails when The task accepts several valid answers and variety matters. Temperature sampling is good for Brainstorming, paraphrasing, costs Unstable output, needs seeds, and fails when Output feeds a parser or a customer-facing fact. Top-p / top-k is good for Trimming the implausible tail, costs Can cut a valid rare token, and fails when The distribution is genuinely flat and the tail holds the answer. Raw model probability as confidence is good for Cheap triage signal, costs Often overconfident, and fails when Used as a user-facing risk score without calibration. Temperature scaling (post-hoc) is good for Calibrating an existing model, costs Needs held-out labelled data, and fails when Traffic distribution shifts after fitting
Failure modes
Confusing confidence with correctness. A model reports 0.92 on an answer that is wrong. Debugging step: bucket predictions by reported confidence, compute empirical accuracy per bucket, and plot the two; the gap is the calibration error.
Multiplying correlated evidence. Three retrieved passages from the same document are treated as three independent confirmations, so the combined confidence is far too high. Debugging step: group evidence by source and count independent sources, not passages.
Base-rate neglect. A classifier with strong sensitivity is deployed on a rare condition and floods reviewers with false positives. Debugging step: compute precision at the actual prevalence, not on the balanced test set.
Underflow in a product of probabilities. A sequence score becomes exactly 0.0 after a few hundred tokens. Debugging step: assert probabilities are stored as log-probabilities and that no score is -inf before a comparison.
In production
Log probabilities, not just decisions. For each model call, record the chosen token's log-probability or the sequence log-probability, then monitor the distribution over time. A shift in mean confidence with flat accuracy is an early warning of distribution change.
Calibrate before you gate. If a threshold routes requests to a human, fit the threshold on labelled data from the same distribution, and re-check it after model or prompt changes. Temperature scaling is a cheap post-hoc fix; it needs a held-out set and does not repair a model that is wrong for structural reasons.
Size your evaluations. Comparing two prompts on twenty examples cannot distinguish a real improvement from noise. Compute the uncertainty on the difference before declaring a winner — this is the same probability arithmetic, applied to your own measurements.
Common misconceptions
People think: A model that says 0.95 is right 95% of the time.
Actually: Only if the model is calibrated. Modern neural networks are frequently overconfident, so reported confidence typically exceeds empirical accuracy until it is calibrated on held-out data.
People think: Two events with probabilities can simply be multiplied to combine them.
Actually: Multiplication requires independence. Model calls, retrieved chunks and judge votes are usually correlated, so the product overstates confidence.
A third: that a probability of 0 means impossibility. In a float32 softmax over a large vocabulary, a token can round to zero while remaining a legitimate continuation. That is a numerical artefact, not a statement about the world.
Interview angle
Probability questions probe whether you can separate three things that look similar: a model's score, a calibrated probability, and a decision threshold. Expect to be asked why softmax needs the max-subtraction, what temperature does to a distribution, how to tell whether a confidence score is trustworthy, and how many evaluation examples you need before believing a result. See logits and softmax and the evaluation questions at /interview/evaluation.
Sources
- Stanford CS229, Probability Theory Review — the definitions used here, with worked derivations: https://cs229.stanford.edu/section/cs229-prob.pdf
- Murphy, Probabilistic Machine Learning: An Introduction — distributions, expectations and Bayesian updating in depth: https://probml.github.io/pml-book/book1.html
- Guo et al., On Calibration of Modern Neural Networks — evidence that deep models are often overconfident, and temperature scaling as a fix: https://arxiv.org/abs/1706.04599
- Goodfellow, Bengio & Courville, Deep Learning, Chapter 3 — probability and information theory for machine learning: https://www.deeplearningbook.org/contents/prob.html