Why it matters
A next-token model has no separate store of what it knows and no flag for uncertainty. When the context lacks an answer, the most probable continuation is still a fluent answer, often assembled from similar entities and numbers. Applications that treat fluency as evidence ship errors that are hard to spot because they look exactly like correct answers, which is why detection has to be structural rather than a matter of reading the output.
Mental model
Hallucination is a confident tour guide in a city they have never visited: the directions are detailed and the street names sound right. The comparison breaks because a guide's hesitation is visible, while a model's uncertainty is flattened into the same fluent register as its facts.
How it works
- 1Separate answerable from unanswerable questions in the evaluation set, so abstention can be scored rather than punished.
- 2Ground answers in retrieved passages and require the model to cite the passage id it used for each claim.
- 3Validate citations mechanically: the cited passage must exist, be in the context, and contain the claim's entities.
- 4Instruct abstention explicitly, with the exact behaviour to use when evidence is missing.
- 5Add a second check for high-risk fields, such as executing code, querying a database or comparing against a source of record.
- 6Track unsupported-claim rate per slice, not overall, because hallucination concentrates in rare entities and recent events.
Step 1 of 6
Separate answerable from unanswerable questions in the evaluation set, so abstention can be scored rather than punished.
Figure. The mechanism in this page's own steps. The figure adds sequence, not new claims.
What to notice. A step can be read alone, but the order is the mechanism. The control lets you stop on any one.
Text version
The page's steps: 1. Separate answerable from unanswerable questions in the evaluation set, so abstention can … 2. Ground answers in retrieved passages and require the model to cite the passage id it used… 3. Validate citations mechanically: the cited passage must exist, be in the context, and con… 4. Instruct abstention explicitly, with the exact behaviour to use when evidence is missing. 5. Add a second check for high-risk fields, such as executing code, querying a database or c… 6. Track unsupported-claim rate per slice, not overall, because hallucination concentrates i…
- Separate answerable from unanswerable questions in the evaluation set, so abstention can be scored rather than punished.
- Ground answers in retrieved passages and require the model to cite the passage id it used for each claim.
- Validate citations mechanically: the cited passage must exist, be in the context, and contain the claim's entities.
- Instruct abstention explicitly, with the exact behaviour to use when evidence is missing.
- Add a second check for high-risk fields, such as executing code, querying a database or comparing against a source of record.
- Track unsupported-claim rate per slice, not overall, because hallucination concentrates in rare entities and recent events.
Trade-offs
| Choice | Good for | Cost | Fails when |
|---|---|---|---|
| Closed-book answer | Fast, no retrieval cost | Unsupported claims on anything rare | The domain requires accuracy |
| Retrieval-grounded answer | Verifiable and fresher | Retrieval misses become refusals or errors | The corpus does not cover the question |
| Strict citation requirement | Auditable claims | More tokens, more refusals, slower | The task is conversational and low stakes |
| Abstain on low confidence | Fewer wrong answers | More gaps for the user | Users expect an attempt on every question |
Good for
Fast, no retrieval cost
Cost
Unsupported claims on anything rare
Fails when
The domain requires accuracy
- The domain requires accuracy
- The corpus does not cover the question
- The task is conversational and low stakes
- Users expect an attempt on every question
Figure. Each choice with what it buys, what it costs, and the case where it loses.
What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.
Text version
Four trade-offs: Closed-book answer is good for Fast, no retrieval cost, costs Unsupported claims on anything rare, and fails when The domain requires accuracy. Retrieval-grounded answer is good for Verifiable and fresher, costs Retrieval misses become refusals or errors, and fails when The corpus does not cover the question. Strict citation requirement is good for Auditable claims, costs More tokens, more refusals, slower, and fails when The task is conversational and low stakes. Abstain on low confidence is good for Fewer wrong answers, costs More gaps for the user, and fails when Users expect an attempt on every question
Failure modes
- Fabricated citation. The answer cites a passage that does not support it or does not exist. Debug: check every citation id against the retrieved set and the quoted span.
- Entity substitution. A number or name is swapped for a similar one from training data. Debug: diff extracted entities against the source text and flag any not present.
- Grounding that never abstains. Retrieval returns something for every query and the model answers from irrelevant context. Debug: include unanswerable questions and measure the abstention rate.
In production
Monitor unsupported-claim rate on a sampled audit and the share of answers with at least one verified citation. Alert when a retrieval outage coincides with a rise in answers, which usually means the model filled the gap.
Common misconceptions
People think: Hallucination is a bug that a bigger model fixes.
Actually: It follows from the training objective; scale reduces some errors and leaves the mechanism intact.
People think: Adding retrieval eliminates hallucination.
Actually: It supplies evidence but the model can still misread, overgeneralise or answer when retrieval returns nothing relevant.
Interview angle
Explain hallucination from the objective, then describe detection as an external check rather than a prompt improvement. Strong answers distinguish covered from uncovered questions and score abstention. Weak answers promise a better prompt and have no mechanism for catching a confident wrong claim.
Sources
- Ji et al., Survey of Hallucination in Natural Language Generation — taxonomy and causes of unsupported generation.
- Lewis et al., Retrieval-Augmented Generation — grounding generation in retrieved documents.
- McKenna et al., Sources of Hallucination by Large Language Models on Inference Tasks — evidence that models rely on prior associations over the prompt.