Skip to content
AI Engineer Guide.

concept · concepts · intermediate · concept 46 of 57

Self/corrective RAG

0 of 57 complete

Why it matters

Retrieval failures are silent by default: a weak set of passages still produces a fluent answer. Self-correction inserts a judgement between retrieval and generation so the system can notice that the evidence does not support the question. The judgement is itself a model call with its own error rate, which is why the loop needs a cap, a fallback and an evaluation that treats a wasted retry as a cost.

Mental model

It is a proofreader who checks the sources before signing off, sending the draft back when a citation does not hold. The metaphor breaks because the proofreader is the same person who wrote the draft, and shares its misreading unless given an independent check.

How it works

Mechanism, step by stepInteractive
  1. 1Grade each retrieved passage for whether it contains evidence that answers the question, not whether it is on topic.
  2. 2Choose an action from the grade: answer, rewrite the query and retrieve again, or abstain with a reason.
  3. 3Rewrite toward the missing attribute rather than paraphrasing the original question again.
  4. 4Regenerate with the new evidence and re-grade the draft's support before returning it.
  5. 5Cap iterations and total retrieval calls, then fall back to the best-supported partial answer or an abstention.
  6. 6Log the grade, action and outcome of every iteration so retries can be evaluated as decisions.

Step 1 of 6

Grade each retrieved passage for whether it contains evidence that answers the question, not whether it is on topic.

Step 1/6 · step 1

Figure. The mechanism in this page's own steps. The figure adds sequence, not new claims.

What to notice. A step can be read alone, but the order is the mechanism. The control lets you stop on any one.

Text version

The page's steps: 1. Grade each retrieved passage for whether it contains evidence that answers the question, … 2. Choose an action from the grade: answer, rewrite the query and retrieve again, or abstain… 3. Rewrite toward the missing attribute rather than paraphrasing the original question again. 4. Regenerate with the new evidence and re-grade the draft's support before returning it. 5. Cap iterations and total retrieval calls, then fall back to the best-supported partial an… 6. Log the grade, action and outcome of every iteration so retries can be evaluated as decis…

  1. Grade each retrieved passage for whether it contains evidence that answers the question, not whether it is on topic.
  2. Choose an action from the grade: answer, rewrite the query and retrieve again, or abstain with a reason.
  3. Rewrite toward the missing attribute rather than paraphrasing the original question again.
  4. Regenerate with the new evidence and re-grade the draft's support before returning it.
  5. Cap iterations and total retrieval calls, then fall back to the best-supported partial answer or an abstention.
  6. Log the grade, action and outcome of every iteration so retries can be evaluated as decisions.

Trade-offs

ChoiceGood forCostFails when
Grade and retry onceCatches weak retrieval cheaplyOne extra retrieval and generationThe first retrieval is usually fine
Iterate until confidentHighest support rateUnbounded latency and costTraffic is interactive
Grade with the same modelNo extra dependencyShared blind spotsThe task requires independent verification
Abstain on weak evidenceFew unsupported answersMore refusalsRecall is mediocre and users need an attempt
Trade-off mapInteractive

Good for

Catches weak retrieval cheaply

Cost

One extra retrieval and generation

Fails when

The first retrieval is usually fine

  1. The first retrieval is usually fine
  2. Traffic is interactive
  3. The task requires independent verification
  4. Recall is mediocre and users need an attempt

Figure. Each choice with what it buys, what it costs, and the case where it loses.

What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.

Text version

Four trade-offs: Grade and retry once is good for Catches weak retrieval cheaply, costs One extra retrieval and generation, and fails when The first retrieval is usually fine. Iterate until confident is good for Highest support rate, costs Unbounded latency and cost, and fails when Traffic is interactive. Grade with the same model is good for No extra dependency, costs Shared blind spots, and fails when The task requires independent verification. Abstain on weak evidence is good for Few unsupported answers, costs More refusals, and fails when Recall is mediocre and users need an attempt

Failure modes

  • Self-agreement. The grader accepts the same wrong passage the generator used, so the loop confirms the error. Debug: compare grader decisions with human labels on a sample and inspect false accepts.
  • Retry without a new query. The same query returns the same passages and the loop burns its budget. Debug: hash the query per iteration and alert when two iterations share a hash.
  • Correction erases a correct answer. A good first answer is regenerated into a worse one because the grader was harsh. Debug: keep both drafts and compare support scores against human judgement.

In production

Track iterations per request, the marginal support gain of each retry and the abstention rate. Cap total model calls per request and alert when the retry rate crosses its budget.

Common misconceptions

People think: Self-correction removes hallucination.

Actually: It reduces unsupported answers when the grader is independent and calibrated; otherwise it can entrench them.

People think: More iterations mean more accuracy.

Actually: Each iteration adds latency and a chance to drift; the useful gain usually comes from the first rewrite.

Interview angle

Describe the grade-decide-act loop, name the grader's own error rate, and bound the iterations. Explain why the rewrite must target the missing attribute. Saying the model checks its answer and retries, with no cap and no independent signal, is the answer that produces loops.

Sources