Skip to content
AI Engineer Guide.

concept · evaluation · intermediate · concept 28 of 28

Error analysis

0 of 28 complete

Why it matters

Teams often choose a metric first and then optimise it, which measures the proxy rather than the product. Error analysis reverses that: read real traces, describe what went wrong in the user's terms, group the descriptions and count them. The largest group becomes the next fix and the next eval cases, and the counts make the priority arguable instead of anecdotal.

Mental model

It is a doctor reading the chart before ordering tests. The metaphor breaks because the chart is a sample, and the disease may be in the patients nobody sampled, so the taxonomy needs counts and a fresh sample later with the same instrument.

How it works

Eighty failures in five root causesInteractive
16 nodes · 6 branches
    • 34
      • 19
      • 9
      • 6
    • 18
      • 11
      • 7
    • 14
      • 9
      • 5
    • 8
      • 8
    • 6
      • 4
      • 2
root causerootruled out or rejectedbar = failures · parent counts include their groups · scaled to the largest group (34)

Step 1 of 7

Read the sample

120 traces sampled at random and read end to end, including 40 successes. The successes are the comparison that tells you whether the pipeline had the answer at all.

root · current

Sample · 120 traces

A random sample read end to end, including successes. The successes are what separate 'the pipeline never had the document' from 'the model used it badly'.

Root of the hierarchy

Step 1/7 · Read the sample

Figure. A taxonomy of 80 sampled failures whose five group counts sum to the sample, with the largest group opened into three slices. Child counts are parts of their parent. The bars are counts, not rates: 34 retrieval misses and 6 format violations are different sizes of problem.

What to notice. Grouping by root cause is what makes the taxonomy actionable; grouping by symptom collapses these five into one bucket called 'answer was wrong'. Counts come from one sample of 120 traces including 40 successes, and carry sampling error the figure does not draw — 8 refusals is a small slice. The numbers are illustrative; the instrument — sample, describe, group, count — is the page's method.

Text version

A failure taxonomy drawn as a tree with counts. The root is a sample of 120 traces: 80 failures and 40 successes read end to end. Five root causes hang from it, and their counts sum to 80. Retrieval missed the answer has 34, split into a prose query at 19, a stale index at 9 and an un-ingested document at 6. Right document, wrong span has 18, split into a chunk boundary splitting a table at 11 and the answer sitting in a later section at 7. Tool call malformed has 14, split into a wrong argument name at 9 and a missing required field at 5. Refused a valid request has 8, all policy over-trigger. Format violation has 6, split into a missing citation at 4 and a wrong unit at 2. Later steps open the retrieval group, convert its dominant cause into eval cases, and re-sample after the fix.

  1. Collect a random sample of real traces including successes, and read them end to end.
  2. Write a one-line description of what went wrong for each failure in the user's terms.
  3. Group the descriptions into root causes and count each group.
  4. Pick the largest group that is fixable with the current architecture.
  5. Turn representative examples into eval cases with a check for that cause.
  6. Re-sample after the fix to see whether the distribution changed.

Trade-offs

ChoiceGood forCostFails when
Small manual sampleDeep understandingSampling errorFailure is rare
Large automated triageCoverageClassifier bias, shallow labelsYou need root causes
Symptom groupingFastDuplicate fixesCauses differ under one symptom
Root-cause groupingActionableSlower, needs judgementLabellers lack system knowledge
Trade-off mapInteractive

Good for

Deep understanding

Cost

Sampling error

Fails when

Failure is rare

  1. Failure is rare
  2. You need root causes
  3. Causes differ under one symptom
  4. Labellers lack system knowledge

Figure. Each choice with what it buys, what it costs, and the case where it loses.

What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.

Text version

Four trade-offs: Small manual sample is good for Deep understanding, costs Sampling error, and fails when Failure is rare. Large automated triage is good for Coverage, costs Classifier bias, shallow labels, and fails when You need root causes. Symptom grouping is good for Fast, costs Duplicate fixes, and fails when Causes differ under one symptom. Root-cause grouping is good for Actionable, costs Slower, needs judgement, and fails when Labellers lack system knowledge

Failure modes

  • Sampling only failures. The base rate and successful patterns are invisible. Debug: include successes in the sample and compare their traces.
  • Symptom taxonomy. Every group is answer was wrong and no fix follows. Debug: rewrite each group label as a cause that names a pipeline stage.
  • Anecdote-driven roadmap. The loudest complaint sets priorities. Debug: compare group counts with the sample size and an interval.

In production

Run error analysis on a fixed cadence and store the taxonomy with counts. Alert when a new group appears after a release.

Common misconceptions

People think: Error analysis is optional once metrics exist.

Actually: Metrics say that quality moved, while analysis says why and what to fix.

People think: More traces always means better analysis.

Actually: Beyond a sample, reading time grows while the dominant causes are already visible.

Interview angle

Sample real traces, group by root cause, count the groups and convert the largest into eval cases. A strong answer includes successes and explains how priority was chosen. Jumping to a metric without reading failures is the weak version.

Sources