Skip to content
AI Engineer Guide.

concept · basics · intermediate · concept 7 of 27

Bayes

0 of 27 complete

Why it matters

A classifier outputs a score, but decisions depend on how common each class actually is. Bayes' theorem makes that dependence explicit: the same likelihood ratio implies different posteriors when the prior changes. This is why a fraud model tuned on a balanced sample fails in production, and why a medical-style screening test with high sensitivity can still be wrong most of the time when the condition is rare.

Mental model

Bayes' theorem is updating a betting position as evidence arrives: you start with odds, the evidence has a known strength, and your new odds are the product. The metaphor breaks because the prior is not a subjective hunch in most ML uses — it is an estimated base rate, and the whole calculation is only as good as that estimate.

How it works

Mechanism, step by stepInteractive
  1. 1Write down the prior probability of each class from the deployment population, not from the training sample.
  2. 2Estimate the likelihood of the observed features under each class, or the class-conditional density.
  3. 3Multiply prior by likelihood for each class to get unnormalised scores.
  4. 4Divide by the sum across classes so the posterior probabilities sum to one.
  5. 5Check calibration by comparing predicted probability with observed frequency in each bin.
  6. 6Recompute the prior when the deployment mix changes, since the likelihood terms usually stay fixed.

Step 1 of 6

Write down the prior probability of each class from the deployment population, not from the training sample.

Step 1/6 · step 1

Figure. The mechanism in this page's own steps. The figure adds sequence, not new claims.

What to notice. A step can be read alone, but the order is the mechanism. The control lets you stop on any one.

Text version

The page's steps: 1. Write down the prior probability of each class from the deployment population, not from t… 2. Estimate the likelihood of the observed features under each class, or the class-condition… 3. Multiply prior by likelihood for each class to get unnormalised scores. 4. Divide by the sum across classes so the posterior probabilities sum to one. 5. Check calibration by comparing predicted probability with observed frequency in each bin. 6. Recompute the prior when the deployment mix changes, since the likelihood terms usually s…

  1. Write down the prior probability of each class from the deployment population, not from the training sample.
  2. Estimate the likelihood of the observed features under each class, or the class-conditional density.
  3. Multiply prior by likelihood for each class to get unnormalised scores.
  4. Divide by the sum across classes so the posterior probabilities sum to one.
  5. Check calibration by comparing predicted probability with observed frequency in each bin.
  6. Recompute the prior when the deployment mix changes, since the likelihood terms usually stay fixed.

Trade-offs

ChoiceGood forCostFails when
Empirical priorMatches the deployment base rateShifts whenever traffic mix changesThe population is unstable or unknown
Uniform priorSimple, no estimationOver-predicts rare classesRates are highly imbalanced
Naive independence assumptionCheap, works with little dataOverconfident when features are correlatedFeatures are strongly dependent
Full probabilistic modelCorrect uncertainty propagationExpensive to specify and fitLatency or data is limited
Trade-off mapInteractive

Good for

Matches the deployment base rate

Cost

Shifts whenever traffic mix changes

Fails when

The population is unstable or unknown

  1. The population is unstable or unknown
  2. Rates are highly imbalanced
  3. Features are strongly dependent
  4. Latency or data is limited

Figure. Each choice with what it buys, what it costs, and the case where it loses.

What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.

Text version

Four trade-offs: Empirical prior is good for Matches the deployment base rate, costs Shifts whenever traffic mix changes, and fails when The population is unstable or unknown. Uniform prior is good for Simple, no estimation, costs Over-predicts rare classes, and fails when Rates are highly imbalanced. Naive independence assumption is good for Cheap, works with little data, costs Overconfident when features are correlated, and fails when Features are strongly dependent. Full probabilistic model is good for Correct uncertainty propagation, costs Expensive to specify and fit, and fails when Latency or data is limited

Failure modes

  • Base rate ignored at serving. A model trained on a balanced sample is deployed on a rare class and floods false positives. Debug: compare predicted positive rate against the known deployment prevalence.
  • Prior double counting. Resampling to balance classes and then applying a prior again shifts probabilities twice. Debug: recompute posteriors from raw scores and compare with the pipeline's output.
  • Correlated features in naive Bayes. Duplicate or derived features multiply the same evidence and produce extreme posteriors. Debug: check posterior confidence against accuracy in each confidence bucket.

In production

Monitor the predicted positive rate against the observed rate in labelled feedback, and recalibrate when the deployment mix changes. Store the prior used at training time with the model so a later recalibration is reproducible.

Common misconceptions

People think: A 99 percent accurate test means a positive result is 99 percent likely to be correct.

Actually: That depends on prevalence: with a rare condition, most positives can be false even at high accuracy, because the prior multiplies the likelihood.

People think: Bayesian reasoning requires subjective beliefs.

Actually: In applied ML the prior is usually an estimated frequency, and the theorem is just the rule for combining it with evidence.

Interview angle

A strong answer writes the rule, identifies each term with a concrete quantity, and works a rare-event example to show why the base rate dominates. It should connect to calibration and to naive Bayes' independence assumption. Weak answers state the formula and then interpret a positive test as the test's accuracy.

Sources