Why it matters
Retrieval failures are silent by default: a weak set of passages still produces a fluent answer. Self-correction inserts a judgement between retrieval and generation so the system can notice that the evidence does not support the question. The judgement is itself a model call with its own error rate, which is why the loop needs a cap, a fallback and an evaluation that treats a wasted retry as a cost.
Mental model
It is a proofreader who checks the sources before signing off, sending the draft back when a citation does not hold. The metaphor breaks because the proofreader is the same person who wrote the draft, and shares its misreading unless given an independent check.
How it works
- 1Grade each retrieved passage for whether it contains evidence that answers the question, not whether it is on topic.
- 2Choose an action from the grade: answer, rewrite the query and retrieve again, or abstain with a reason.
- 3Rewrite toward the missing attribute rather than paraphrasing the original question again.
- 4Regenerate with the new evidence and re-grade the draft's support before returning it.
- 5Cap iterations and total retrieval calls, then fall back to the best-supported partial answer or an abstention.
- 6Log the grade, action and outcome of every iteration so retries can be evaluated as decisions.
Step 1 of 6
Grade each retrieved passage for whether it contains evidence that answers the question, not whether it is on topic.
Figure. The mechanism in this page's own steps. The figure adds sequence, not new claims.
What to notice. A step can be read alone, but the order is the mechanism. The control lets you stop on any one.
Text version
The page's steps: 1. Grade each retrieved passage for whether it contains evidence that answers the question, … 2. Choose an action from the grade: answer, rewrite the query and retrieve again, or abstain… 3. Rewrite toward the missing attribute rather than paraphrasing the original question again. 4. Regenerate with the new evidence and re-grade the draft's support before returning it. 5. Cap iterations and total retrieval calls, then fall back to the best-supported partial an… 6. Log the grade, action and outcome of every iteration so retries can be evaluated as decis…
- Grade each retrieved passage for whether it contains evidence that answers the question, not whether it is on topic.
- Choose an action from the grade: answer, rewrite the query and retrieve again, or abstain with a reason.
- Rewrite toward the missing attribute rather than paraphrasing the original question again.
- Regenerate with the new evidence and re-grade the draft's support before returning it.
- Cap iterations and total retrieval calls, then fall back to the best-supported partial answer or an abstention.
- Log the grade, action and outcome of every iteration so retries can be evaluated as decisions.
Trade-offs
| Choice | Good for | Cost | Fails when |
|---|---|---|---|
| Grade and retry once | Catches weak retrieval cheaply | One extra retrieval and generation | The first retrieval is usually fine |
| Iterate until confident | Highest support rate | Unbounded latency and cost | Traffic is interactive |
| Grade with the same model | No extra dependency | Shared blind spots | The task requires independent verification |
| Abstain on weak evidence | Few unsupported answers | More refusals | Recall is mediocre and users need an attempt |
Good for
Catches weak retrieval cheaply
Cost
One extra retrieval and generation
Fails when
The first retrieval is usually fine
- The first retrieval is usually fine
- Traffic is interactive
- The task requires independent verification
- Recall is mediocre and users need an attempt
Figure. Each choice with what it buys, what it costs, and the case where it loses.
What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.
Text version
Four trade-offs: Grade and retry once is good for Catches weak retrieval cheaply, costs One extra retrieval and generation, and fails when The first retrieval is usually fine. Iterate until confident is good for Highest support rate, costs Unbounded latency and cost, and fails when Traffic is interactive. Grade with the same model is good for No extra dependency, costs Shared blind spots, and fails when The task requires independent verification. Abstain on weak evidence is good for Few unsupported answers, costs More refusals, and fails when Recall is mediocre and users need an attempt
Failure modes
- Self-agreement. The grader accepts the same wrong passage the generator used, so the loop confirms the error. Debug: compare grader decisions with human labels on a sample and inspect false accepts.
- Retry without a new query. The same query returns the same passages and the loop burns its budget. Debug: hash the query per iteration and alert when two iterations share a hash.
- Correction erases a correct answer. A good first answer is regenerated into a worse one because the grader was harsh. Debug: keep both drafts and compare support scores against human judgement.
In production
Track iterations per request, the marginal support gain of each retry and the abstention rate. Cap total model calls per request and alert when the retry rate crosses its budget.
Common misconceptions
People think: Self-correction removes hallucination.
Actually: It reduces unsupported answers when the grader is independent and calibrated; otherwise it can entrench them.
People think: More iterations mean more accuracy.
Actually: Each iteration adds latency and a chance to drift; the useful gain usually comes from the first rewrite.
Interview angle
Describe the grade-decide-act loop, name the grader's own error rate, and bound the iterations. Explain why the rewrite must target the missing attribute. Saying the model checks its answer and retries, with no cap and no independent signal, is the answer that produces loops.
Sources
- Asai et al., Self-RAG — learning to critique retrieved evidence and generation with reflection tokens.
- Yan et al., Corrective Retrieval Augmented Generation — grading retrieval quality and correcting before generating.
- Es et al., RAGAS — reference-free measures of context relevance and faithfulness for such loops.