Why it matters
An answer without evidence forces the reader to trust the system. That is acceptable for low-stakes text and unacceptable for support replies, medical summaries, contract review or anything an auditor will read. Citations change the review cost: instead of re-deriving the answer, the reader checks a highlighted sentence.
They also change debugging. When a wrong answer carries a citation, you can tell within seconds whether retrieval surfaced bad evidence, the model misread good evidence, or the citation itself was fabricated. Without citations, all three look identical from the outside.
The catch is that a citation is only worth what its validation is worth. A model that emits a document title after every sentence produces the appearance of rigour while adding no verifiable information.
Mental model
A citation is a footnote in a research memo, written by someone who actually opened the page. The reader can follow the number, read the quoted sentence, and decide for themselves.
The analogy breaks because the author here is a language model that generates the footnote as fluently as the claim. It can cite a real document it was shown without the passage supporting the sentence, or reconstruct a quote from memory of similar text. So the footnote must be checked by something other than the writer — a string match for the quote, and a separate judgement for the support relationship.
How it works
A citation is an edge from a claim to evidence. Document-level edges are cheap and weak; span-level edges are what a reader can check.
Step 1 of 6
Read the shape
Three claims and three documents. The citation edges are pointers; the return edge is a quote found verbatim, with support judged separately; the conflict edges are the two failures that look like citations until checked.
Narrow layout: nodes in reading order, each followed by its outgoing edges. The wide drawing shows the same graph as geometry.
claim
3 connections
Claim 1
refund within 14 days
A claim with a quoted span that exists in the cited passage and supports it. This is the edge a reader can follow: claim, quote, document, offset.
No outgoing edge.
document · claim · 7 nodes · 9 edges · fixed authored layout, no simulation · list
Figure. A citation is an edge from a claim to evidence, and its granularity decides whether a reader can check it. Claim 1 points at a quoted span in a supplied help article and the quote is found verbatim, with support judged separately; Claim 2 points at a real passage about another plan, so existence passes and support fails; Claim 3 cites a chunk that was retrieved and then dropped from the prompt.
What to notice. The same pointer shape carries a verified quote, an unsupported claim and an impossible citation, so the edge alone is not evidence of grounding. The figure omits retrieval scores, the assembly budget and the validator; it shows attribution at span level, not a bibliography. Document names, section numbers and offsets are illustrative.
Text version
Seven nodes. An answer extracts three claims. Claim 1 cites the help article at section 4.2, offset 412, and a return edge confirms the quote was found. Claim 2 cites the policy page at section 1, offset 88, and a conflict edge from the page says it supports another plan, not this claim. Claim 3 cites a dropped chunk at section 7, offset 40, and a conflict parallel edge says it was not in the prompt. Steps light the whole graph, the per-claim pointers, the span check, the unsupported real quote, the unseen chunk, and the fail-closed policy.
- Give every candidate a stable ID. The context assembly step labels each passage with an ID, title, section and date. IDs must be the same ones the application can resolve back to a document and offset.
- Ask for structured output. Rather than "cite your sources", request claims where each entry has the claim text, the supporting chunk ID, and a verbatim quote. Structured output makes verification possible; free-form markers do not.
- Constrain to supplied evidence. Only passages in the context may be cited, and an unsupported claim must be dropped or marked unknown. Give an explicit abstain path.
- Verify the quote exists. A substring check against the cited chunk catches fabrication and paraphrase-as-quote. Normalise whitespace before comparing, and keep offsets for highlighting.
- Verify support. A quote can exist and still not support the claim — typically a passage about a different plan or an older policy. Run an entailment check, either an NLI model or a judge given the claim and the passage only.
- Render at the right granularity. Inline markers at the sentence or clause level, with a hover card showing the quote and a link to the source at the correct offset. A single list of documents at the end is a bibliography, not attribution.
- Fail closed. If a sentence has no verified support, either remove it or label it explicitly as unverified in the UI.
- Granularity
- Claim or sentence level
- Minimum viable check
- Quoted span exists verbatim
- Stronger check
- Passage entails the claim
- Metrics
- Citation recall and precision
- Common failure
- Real citation, wrong claim
Measuring citation quality
Two numbers, in the spirit of the ALCE formulation:
Recall penalises uncited assertions; precision penalises noise and misattribution. Track both. A system that cites every sentence scores high on recall and can be nearly useless if precision is low, and a system that cites nothing has undefined precision and near-zero recall on anything that matters.
A minimal validator
def verify_citations(answer, chunks_by_id, entails) -> list[dict]:
"""Mechanical checks first, semantic check second, fail closed."""
results = []
for claim in answer.claims:
chunk = chunks_by_id.get(claim.chunk_id)
if chunk is None:
results.append({"claim": claim.text, "ok": False, "why": "unknown chunk"})
continue
if normalise(claim.quote) not in normalise(chunk.text):
results.append({"claim": claim.text, "ok": False, "why": "quote not in chunk"})
continue
if not entails(claim.text, chunk.text): # NLI model or judge
results.append({"claim": claim.text, "ok": False, "why": "not supported"})
continue
results.append({"claim": claim.text, "ok": True, "chunk_id": claim.chunk_id})
return resultsRun the mechanical checks on every request — they are microseconds. The semantic check is a model call, so it is often sampled, batched, or run only for high-stakes intents; decide by stakes, not by average latency.
Trade-offs
| Approach | Good for | Cost | Fails when |
|---|---|---|---|
| Document-level attribution | Broad research, provenance only | One title per answer | The reader needs to verify a specific number |
| Prompted inline markers | Cheap first version | Model tokens, no verification | The model cites a passage that does not support the claim |
| Structured quotes plus string check | Most product surfaces | Schema, offset storage, one pass | The quote supports a different claim than the one made |
| Post-hoc attribution model | Answers generated without citations | Extra model call per sentence | The generator paraphrased beyond the passage |
| Full verification on every claim | Legal, medical, regulated answers | Latency and judge cost per request | Traffic is high-volume and low-stakes |
Good for
Broad research, provenance only
Cost
One title per answer
Fails when
The reader needs to verify a specific number
- The reader needs to verify a specific number
- The model cites a passage that does not support the claim
- The quote supports a different claim than the one made
- The generator paraphrased beyond the passage
- Traffic is high-volume and low-stakes
Figure. Each choice with what it buys, what it costs, and the case where it loses.
What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.
Text version
Four trade-offs: Document-level attribution is good for Broad research, provenance only, costs One title per answer, and fails when The reader needs to verify a specific number. Prompted inline markers is good for Cheap first version, costs Model tokens, no verification, and fails when The model cites a passage that does not support the claim. Structured quotes plus string check is good for Most product surfaces, costs Schema, offset storage, one pass, and fails when The quote supports a different claim than the one made. Post-hoc attribution model is good for Answers generated without citations, costs Extra model call per sentence, and fails when The generator paraphrased beyond the passage. Full verification on every claim is good for Legal, medical, regulated answers, costs Latency and judge cost per request, and fails when Traffic is high-volume and low-stakes
Failure modes
- Fabricated quote. The citation points at a real document but the quoted sentence does not exist in it. Debug: substring-match every quote against its cited chunk and log the failure rate; this is the cheapest check in the system and should always be on.
- Unsupported but real. The quote exists and is about the same topic, but does not establish the claim — often because the passage describes a different plan, region or date. Debug: sample 50 verified citations and have a human judge label support; if the rate is low, tighten the generation instruction and add an entailment check.
- Citation to an unseen chunk. The answer cites a passage that was retrieved but dropped during assembly. Debug: intersect cited IDs with the IDs actually placed in the prompt; the difference is invisible to the model but catastrophic for trust.
- Stale offsets. Re-chunking shifts character positions and highlights land on the wrong sentence. Debug: store document version with each citation and re-resolve offsets at render time; assert that the quote still matches the slice.
In production
Verification is the cost centre. Exact-match checks are free; entailment calls can approach the price of generation, so most teams run them on a sample or on flagged intents, relying on offline evaluation for the rest. Cache verification per (claim, chunk) pair so retries and re-renders do not pay twice.
Monitor citation coverage (claims with a verified citation), quote-match rate, judge-verified support rate, and click-through on citation links. Falling coverage usually means retrieval is returning weak evidence, not that the generator changed.
Common misconceptions
People think: If the answer has citations, it is grounded.
Actually: Citations are generated text too. Without validation, a model can attach a real document to a claim it does not support, and readers rarely check.
People think: Document-level citations are enough for a product.
Actually: A title tells the reader where to start reading. Attribution that saves review time points at the sentence and shows the quote.
Interview angle
The question tends to be "how do you know the citation is correct", and the weak answer is "we ask the model to cite". The strong answer separates existence from support, describes mechanical verification plus an entailment check, and names citation precision and recall as the metrics. A good follow-up to prepare: what happens to citations after a document is updated. See RAG interview questions.
Sources
- Gao et al., Enabling Large Language Models to Generate Text with Citations — the ALCE benchmark, which scores citations with recall and precision against human-written references.
- Bohnet et al., Attributed Question Answering: Evaluation and Modeling for Attributed Large Language Models — evaluating attribution separately from answer correctness.
- Anthropic, Citations — an API design where citations are returned as spans grounded in supplied documents.
- Liu et al., Lost in the Middle — why the position of cited evidence in a long context affects whether the model uses it.