Skip to content
AI Engineer Guide.
seniorPracticecurated

Answers cite real documents that do not actually contain the claim. How do you detect and fix bad citations?

30-second answer

Likely follow-ups

Open a question to rehearse the answer.

When verification finds an unsupported citation, do you strip the marker or regenerate the answer?
  • Strip when the claim is peripheral and the rest of the answer stands, so the marker goes and the sentence is removed or shown as unverified.
  • Regenerate when the claim is central and the evidence was in the prompt, passing the failing claim and its cited span back so the repair is targeted.
  • Never add a citation the checker did not verify, because a repair that invents support is worse than an abstention.
How do you score citation accuracy without human review on every release?
  • Compute citation precision and recall mechanically on a labelled set where each claim has a known supporting span.
  • Run the quote-existence check on all traffic because it costs microseconds, and sample the entailment check by stakes rather than by average.
  • Keep a small human-labelled slice to calibrate the judge, and re-baseline when the judge model or prompt changes.
How do you stop near-duplicate policy versions from being confused when both are retrieved?
  • Store version, effective date and supersedes links on every chunk, then filter to the version in force instead of trusting wording similarity.
  • Delete superseded chunks at ingest rather than leaving both retrievable, and reconcile the index against source versions.
  • When an archived copy must stay for audit, mark it non-retrievable for answering and cite the current version.

Red flags

  • citation implies grounding
  • existence-only check

Concepts this tests

Tags

More in RAG & retrieval pipelines