Skip to content
AI Engineer Guide.

concept · evaluation · intermediate · concept 10 of 28

Judge bias

0 of 28 complete

Why it matters

A model judge is a measurement instrument with known distortions. It tends to favour the first candidate, longer answers, confident phrasing and text that resembles its own generations. If a system's outputs differ in length or style from its competitor's, the judge's preference becomes a confound that looks like a quality difference. Comparing systems without measuring those distortions produces a ranking that may reverse when the presentation changes.

Mental model

It is a referee who favours the home team. The metaphor breaks because this referee also wrote one team's playbook and finds that style familiar, so the preference looks like expertise rather than bias to anyone reading the score, and nobody audits the referee.

How it works

Position bias · identical answers, two ordersInteractive

hover or focus a cell

0%100%
presentation order ↓judge verdict →
A winsB winstieshare
A first100%
B first100%

Hover or focus a cell to read its exact value and printed row total. Shade and printed number encode the same quantity.

first position wins
124/200 = 62%
second position wins
36/200 = 18%
ties
40/200 = 20%
candidate A overall
80/200 = 40%

Step 1 of 4

Both orders

The same pair is judged twice, once with A first and once with B first, over 100 pairs. The candidate text is identical in both arms.

Next: A shown first

Step 1/4 · Both orders

Figure. An illustrative position-bias probe with the two candidate answers held identical. Each row is one presentation order over 100 pairs; the three columns are the judge's verdict. With identical answers and a tie option, an unbiased judge would call a tie every time; a judge that chose anyway would split near 50/50. Here the first slot wins 62% of pairs in both orders and 20% of pairs are ties.

What to notice. The rows are the control: the answers are the same, only their order changes. Candidate A wins 62% when shown first and 18% when shown second, so its 40% overall win rate averages a large first-position advantage with a large second-position penalty, not a stable quality difference. The 20% tie rate is not the null result: on identical answers ties are the honest verdict, and the bias signal is the 62/18 split that survives the order swap.

Text version

A two by three matrix with rows A first and B first and columns A wins, B wins and tie. When A is first, A wins 62%, B wins 18% and 20% tie. When B is first, A wins 18%, B wins 62% and 20% tie. Row totals read 100%. Steps highlight each order and then the average. The readout reports the order, the verdict and the share of the 100 pairs.

  1. Score the same pair in both orders and count how often the verdict flips.
  2. Pad a known-worse answer and check whether its score rises with length alone.
  3. Grade outputs from the judge's own model family against others to measure self-preference.
  4. Render both candidates with the same template and a fixed length budget.
  5. Aggregate across several judges or randomise judge assignment per item.
  6. Report the bias probe results next to the score they qualify.

Trade-offs

ChoiceGood forCostFails when
Single judgeCheap and consistentOne model's biases dominateScores decide a release
Panel of judgesReduces idiosyncratic biasCost multipliesJudges share the same bias
Pairwise comparisonAvoids absolute scale driftPosition effects, no absolute scoreYou need calibrated thresholds
Absolute rubric scoreComparable over timeScale interpretation driftsJudge recalibrates mid-run
Trade-off mapInteractive

Good for

Cheap and consistent

Cost

One model's biases dominate

Fails when

Scores decide a release

  1. Scores decide a release
  2. Judges share the same bias
  3. You need calibrated thresholds
  4. Judge recalibrates mid-run

Figure. Each choice with what it buys, what it costs, and the case where it loses.

What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.

Text version

Four trade-offs: Single judge is good for Cheap and consistent, costs One model's biases dominate, and fails when Scores decide a release. Panel of judges is good for Reduces idiosyncratic bias, costs Cost multiplies, and fails when Judges share the same bias. Pairwise comparison is good for Avoids absolute scale drift, costs Position effects, no absolute score, and fails when You need calibrated thresholds. Absolute rubric score is good for Comparable over time, costs Scale interpretation drifts, and fails when Judge recalibrates mid-run

Failure modes

  • Position win. The first candidate wins regardless of content. Debug: swap positions and count verdict flips.
  • Length inflation. Longer answers score higher after controlling for correctness. Debug: regress score on length and correctness jointly.
  • Self-preference. A judge ranks its own family's outputs above human preference. Debug: include a cross-family judge and compare rankings.

In production

Run bias probes on every judge or prompt change and store the results. Cap answer length in comparisons and randomise order per item.

Common misconceptions

People think: A stronger model makes an unbiased judge.

Actually: Capability and bias are separate, and stronger judges still show position and self-preference effects.

People think: Bias cancels out across many items.

Actually: It cancels only if it is independent of the condition, while correlated bias shifts the comparison.

Interview angle

Name each bias, describe a probe and explain how the design neutralises it. A strong answer says why bias correlated with the condition does not average out. Claiming a good prompt removes bias, with no measurement of order or length, is the weak response.

Sources