Skip to content
AI Engineer Guide.

concept · concepts · beginner · concept 3 of 12

Predictive vs generative

0 of 12 complete

Why it matters

A churn classifier returns a number in a known range, so you can measure calibration, choose a threshold and bound the damage of an error. A model writing a customer reply can produce anything, including a correct-sounding falsehood, and no threshold contains it. Treating the second like the first produces evaluation that measures nothing and guardrails that arrive too late.

Mental model

A predictive model is a multiple-choice exam; a generative model is an essay question. The metaphor breaks because an essay is marked by a reader who understands the subject, while automated grading of generated text is itself a model judgement with its own error.

How it works

Mechanism, step by stepInteractive
  1. 1State the output contract: a label set, a numeric range, or a specification for the generated artefact.
  2. 2For predictive tasks, build a labelled set and measure calibration and per-class errors.
  3. 3For generative tasks, define a rubric and collect reference outputs where possible.
  4. 4Constrain generation with schemas, allowed values and post-generation validation.
  5. 5Evaluate both with slices, since aggregate scores hide the class or topic that fails.
  6. 6Decide where a deterministic check can replace a model judgement, and put it first.

Step 1 of 6

State the output contract: a label set, a numeric range, or a specification for the generated artefact.

Step 1/6 · step 1

Figure. The mechanism in this page's own steps. The figure adds sequence, not new claims.

What to notice. A step can be read alone, but the order is the mechanism. The control lets you stop on any one.

Text version

The page's steps: 1. State the output contract: a label set, a numeric range, or a specification for the gener… 2. For predictive tasks, build a labelled set and measure calibration and per-class errors. 3. For generative tasks, define a rubric and collect reference outputs where possible. 4. Constrain generation with schemas, allowed values and post-generation validation. 5. Evaluate both with slices, since aggregate scores hide the class or topic that fails. 6. Decide where a deterministic check can replace a model judgement, and put it first.

  1. State the output contract: a label set, a numeric range, or a specification for the generated artefact.
  2. For predictive tasks, build a labelled set and measure calibration and per-class errors.
  3. For generative tasks, define a rubric and collect reference outputs where possible.
  4. Constrain generation with schemas, allowed values and post-generation validation.
  5. Evaluate both with slices, since aggregate scores hide the class or topic that fails.
  6. Decide where a deterministic check can replace a model judgement, and put it first.

Trade-offs

ChoiceGood forCostFails when
Predictive modelCalibrated, cheap, easy to thresholdOnly answers predefined questionsThe output space is open
Generative modelHandles open-ended input and outputEvaluation is indirect and costlyThe decision must be a fixed label
Generator with schemaStructured, validatable outputConstrains expression and can truncateThe task needs free-form prose
Classifier plus generatorCheap routing, rich explanationTwo systems and a join between themConsistency between the two matters
Trade-off mapInteractive

Good for

Calibrated, cheap, easy to threshold

Cost

Only answers predefined questions

Fails when

The output space is open

  1. The output space is open
  2. The decision must be a fixed label
  3. The task needs free-form prose
  4. Consistency between the two matters

Figure. Each choice with what it buys, what it costs, and the case where it loses.

What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.

Text version

Four trade-offs: Predictive model is good for Calibrated, cheap, easy to threshold, costs Only answers predefined questions, and fails when The output space is open. Generative model is good for Handles open-ended input and output, costs Evaluation is indirect and costly, and fails when The decision must be a fixed label. Generator with schema is good for Structured, validatable output, costs Constrains expression and can truncate, and fails when The task needs free-form prose. Classifier plus generator is good for Cheap routing, rich explanation, costs Two systems and a join between them, and fails when Consistency between the two matters

Failure modes

  • Classification metric on generated text. Exact match scores near zero and says nothing about quality. Debug: check whether the metric distinguishes a good paraphrase from a wrong answer.
  • Unbounded output. The generator emits content outside the intended form or policy. Debug: validate every output against the schema and sample the rejects.
  • Label set drift. New categories appear in production and the classifier forces them into old ones. Debug: inspect the confidence distribution and the share of low-confidence predictions.

In production

For predictive models track calibration and per-class error; for generative ones track schema validity, refusal rate and rubric scores on a sample. Both need input drift monitoring.

Common misconceptions

People think: Generative models replace predictive ones.

Actually: A classifier is cheaper, calibrated and easier to audit when the question is closed; generation is for open output.

People think: Generative output cannot be evaluated.

Actually: It can, with rubrics, references and deterministic checks on form; the evaluation is just less direct.

Interview angle

Strong answers start from the output space and derive the evaluation and constraints from it, then say when a classifier is the better tool. Weak answers treat generation as a strictly newer version of prediction.

Sources