Skip to content
AI Engineer Guide.

concept · concepts · intermediate · concept 16 of 22

Synthetic data

0 of 22 complete

Why it matters

Instruction tuning and preference tuning both want more coverage than annotators can supply. A capable model can draft examples across many task types, which makes dataset size a budget question rather than a scheduling one. The risk is that the generator's errors become the student's ground truth, and that repeated generation on model output narrows the distribution instead of widening it.

Mental model

Synthetic data is photocopying a photocopy: fast, convenient, and each generation loses a little definition until the text is no longer readable. The metaphor breaks because you can inspect each copy and mix it back with the originals, which is the practice that keeps a synthetic set usable.

How it works

Mechanism, step by stepInteractive
  1. 1Write a seed set of human examples that define the behaviour and format you want replicated.
  2. 2Generate variations with an explicit schema and a diversity instruction, sampling across categories rather than one prompt template.
  3. 3Filter aggressively: verify factual claims against a source, run executable checks where possible, and discard refusals and malformed records.
  4. 4Deduplicate against both the seed set and the generated set, since generators repeat themselves.
  5. 5Mix synthetic and human data rather than replacing one with the other, and record the ratio.
  6. 6Evaluate on human-written held-out prompts only, so the score is not measuring the generator.

Step 1 of 6

Write a seed set of human examples that define the behaviour and format you want replicated.

Step 1/6 · step 1

Figure. The mechanism in this page's own steps. The figure adds sequence, not new claims.

What to notice. A step can be read alone, but the order is the mechanism. The control lets you stop on any one.

Text version

The page's steps: 1. Write a seed set of human examples that define the behaviour and format you want replicat… 2. Generate variations with an explicit schema and a diversity instruction, sampling across … 3. Filter aggressively: verify factual claims against a source, run executable checks where … 4. Deduplicate against both the seed set and the generated set, since generators repeat them… 5. Mix synthetic and human data rather than replacing one with the other, and record the rat… 6. Evaluate on human-written held-out prompts only, so the score is not measuring the genera…

  1. Write a seed set of human examples that define the behaviour and format you want replicated.
  2. Generate variations with an explicit schema and a diversity instruction, sampling across categories rather than one prompt template.
  3. Filter aggressively: verify factual claims against a source, run executable checks where possible, and discard refusals and malformed records.
  4. Deduplicate against both the seed set and the generated set, since generators repeat themselves.
  5. Mix synthetic and human data rather than replacing one with the other, and record the ratio.
  6. Evaluate on human-written held-out prompts only, so the score is not measuring the generator.

Trade-offs

ChoiceGood forCostFails when
Fully syntheticScale and coverageInherited errors and mode collapseCorrectness is the product
Synthetic plus human seedBroad with an anchorMore pipeline workYou cannot verify outputs
Self-distillation roundsCheap difficulty escalationAmplifies existing failuresThe base model is already wrong
Human onlyTrustworthy targetsSlow and expensiveCoverage is the bottleneck
Trade-off mapInteractive

Good for

Scale and coverage

Cost

Inherited errors and mode collapse

Fails when

Correctness is the product

  1. Correctness is the product
  2. You cannot verify outputs
  3. The base model is already wrong
  4. Coverage is the bottleneck

Figure. Each choice with what it buys, what it costs, and the case where it loses.

What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.

Text version

Four trade-offs: Fully synthetic is good for Scale and coverage, costs Inherited errors and mode collapse, and fails when Correctness is the product. Synthetic plus human seed is good for Broad with an anchor, costs More pipeline work, and fails when You cannot verify outputs. Self-distillation rounds is good for Cheap difficulty escalation, costs Amplifies existing failures, and fails when The base model is already wrong. Human only is good for Trustworthy targets, costs Slow and expensive, and fails when Coverage is the bottleneck

Failure modes

  • Model collapse. Later generations are less diverse and the student parrots one style. Debug: measure distinct-n or embedding spread per generation round and compare with the seed set.
  • Error amplification. A systematic mistake appears in most synthetic answers and becomes the student's belief. Debug: sample by category and check claims against a trusted source, not against another model.
  • Contamination. Synthetic evaluation items were generated from the same prompts as training. Debug: hash prompts and keep evaluation prompts human-authored and disjoint.

In production

Tag every record with its generator and round, and keep the ability to retrain without a given batch. Monitor output diversity and refusal rate after each synthetic-heavy run.

Common misconceptions

People think: Synthetic data is free data.

Actually: Verification and filtering usually cost more than generation.

People think: A stronger generator fixes quality.

Actually: A stronger generator produces more convincing errors, which are harder to filter.

Interview angle

Treat generation as the cheap half and verification as the work. Name diversity collapse and error amplification, and say evaluation must use human prompts. Weak answers describe a pipeline that generates and trains with no filtering step and no provenance.

Sources