Skip to content
AI Engineer Guide.

concept · concepts · intermediate · concept 38 of 57

Grounded generation

0 of 57 complete

Why it matters

A retrieved passage in the prompt is an offer, not a constraint. Models routinely answer from pretraining when the context is thin, and the result is fluent, plausible and unverifiable. Grounded generation is the set of instructions and output constraints that make the answer traceable to specific text: what to cite, how to phrase uncertainty and what to do when the passages disagree.

Mental model

Grounded generation is a witness testifying with documents in hand: every claim should be traceable to a page the court has seen. The metaphor breaks because the witness has read widely before the trial and cannot always separate what it read from what it remembers.

How it works

The answer as a record of spansInteractive

Synthetic answer record · query r-2210 · 3 passages

Four sentences · three spans · one abstention rule

bounding boxes

Step 1/6 · A passage is an offer

Step 1A passage is an offer

Retrieval puts three passages in the prompt, and nothing forces their use. The answer can be fluent, correct-sounding and drawn entirely from memory.

GROUNDED ANSWER · QUERY r-2210 · PASSAGES p3, p7, p9

, running header, region 1. Select to inspect.

Every sentence carries a span

, title, region 2. Select to inspect.
INSTRUCTION
Use only passages p3, p7, p9. Cite the passage label after each factual sentence. Quote the span before paraphrasing it. If the passages disagree, report both, with labels. If they do not answer, say so and stop.
, code block INSTRUCTION, region 3. Select to inspect.
Answer

Annual plans moved to a single renewal date in March [p3]. Monthly plans kept their cycle; the window narrowed from 14 to 7 days [p3]. A later notice gives 10 days, so the two pages disagree [p7]. Most churned accounts were on the legacy plan.

, paragraph Answer, region 4. Select to inspect.
CITATION TRACE
#CiteSpanSupport
S1p3137–198supported
S2p3214–298supported
S3p788–131supported
S4——✕uncited
, table CITATION TRACE, region 5. Select to inspect.
ConflictWindow: p3 gives 7 days · p7 gives 10 days · both reported
, form field Conflict, region 6. Select to inspect.
Abstentionno supporting passage → abstain · 1 of 40 test questions
, form field Abstention, region 7. Select to inspect.
CHECKS
cited 3 · supported 3 · uncited 1 spans resolved 3 · quotes outside span 0 verdict fail · sentence 4
, code block CHECKS, region 8. Select to inspect.

4 sentences · 3 cited · 1 unsupported · 1 conflict · 0 outside-span quotes

, running footer, region 9. Select to inspect.
9 regions

Step record

Passages
3
Instruction
explicit
Checked
not yet

The text version summarises this figure; select a region for its full text.

Markscheck failed

Figure. Four sentences over three admitted passages. Three carry a citation that resolves to a character span; the fourth carries none, so the answer fails its own check. The two passages disagree on the cancellation window and both are reported. Abstention is a defined output, tested on one of forty questions.

What to notice. Synthetic answer and passages. Sentence text, spans and scores are authored; the arithmetic of cited and uncited sentences is exact for what is shown. The figure shows the record a checker can verify, not a model's measured faithfulness.

Text version

A grounded answer record for query r-2210 over passages p3, p7 and p9. The instruction requires citations after factual sentences, quotes before paraphrase, a report of conflicts and an explicit abstention. The answer has four sentences: two cite p3, one cites p7 and reports that the window is given as 7 days on p3 and 10 days on p7, and one about churned accounts has no citation. A trace table gives spans 137–198, 214–298 and 88–131 for the three cited sentences, and a dash for the uncited one. Fields record the conflict and an abstention on 1 of 40 test questions. A check block reports 3 cited, 3 supported, 1 uncited, 0 quotes outside their span and a fail verdict on sentence 4. Steps mark the instruction, the citations, the spans, the conflict, abstention and the check.

  1. Instruct the model to use only the supplied passages, and to say the passages do not answer when they do not.
  2. Require a citation marker after each factual sentence, pointing at a passage label that exists in the prompt.
  3. Ask for a short verbatim quote before each paraphrase where the domain needs auditability.
  4. Define conflict handling: prefer a stated authority or recency order, or report both claims with their sources.
  5. Constrain the output shape so an answer without citations is a detectable failure rather than a stylistic choice.
  6. Evaluate on questions with no supporting passage, since abstention is the behaviour most often untested.

Trade-offs

ChoiceGood forCostFails when
Strict quote-only answersMaximum auditabilityStilted output and long promptsUsers need synthesis across passages
Citation per sentenceCheap to verifyVerbose and sometimes mechanicalThe answer is a single short fact
Allow parametric knowledgeHandles gaps gracefullyUnverifiable claims enter the answerThe domain is regulated or safety-critical
Abstain readilyLow hallucination rateMore unanswered questionsRetrieval recall is mediocre and users expect an attempt
Trade-off mapInteractive

Good for

Maximum auditability

Cost

Stilted output and long prompts

Fails when

Users need synthesis across passages

  1. Users need synthesis across passages
  2. The answer is a single short fact
  3. The domain is regulated or safety-critical
  4. Retrieval recall is mediocre and users expect an attempt

Figure. Each choice with what it buys, what it costs, and the case where it loses.

What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.

Text version

Four trade-offs: Strict quote-only answers is good for Maximum auditability, costs Stilted output and long prompts, and fails when Users need synthesis across passages. Citation per sentence is good for Cheap to verify, costs Verbose and sometimes mechanical, and fails when The answer is a single short fact. Allow parametric knowledge is good for Handles gaps gracefully, costs Unverifiable claims enter the answer, and fails when The domain is regulated or safety-critical. Abstain readily is good for Low hallucination rate, costs More unanswered questions, and fails when Retrieval recall is mediocre and users expect an attempt

Failure modes

  • Answer from memory. The model states a fact absent from every passage and cites a plausible-looking one. Debug: for each cited sentence, check the quoted span exists in the cited passage.
  • Citation drift. Citations are present but point at the wrong passage. Debug: score each citation by whether the passage entails the sentence, not by whether a marker exists.
  • Conflict flattened. Two passages disagree and the answer presents one as fact. Debug: search the admitted passages for contradictory values on the same attribute and inspect how the answer handled them.

In production

Track the rate of uncited sentences and the abstention rate on an unanswerable set. Cap the number of passages so citation checking stays tractable, and sample answers weekly for support.

Common misconceptions

People think: Retrieval-augmented means grounded.

Actually: Retrieval supplies context; grounding is a behaviour that must be instructed, constrained and checked.

People think: Abstention is a failure.

Actually: On an unanswerable question, abstaining is the correct answer and a hallucinated answer is the failure.

Interview angle

Separate supplying context from enforcing its use, then describe citation constraints and an abstain path. Name the unanswerable evaluation set. Saying the model is given the documents and will therefore use them is precisely the assumption that produces hallucination.

Sources