Skip to content
AI Engineer Guide.

concept · concepts · intermediate · concept 16 of 57

Retrieval fusion (RRF)

0 of 57 complete

Why it matters

Lexical and dense retrievers miss different queries: one loses paraphrases, the other loses identifiers. Running both and merging their output recovers recall without choosing a winner, but the merge itself is a design decision. Raw scores from different scoring functions are not comparable, so naive addition lets whichever retriever has the wider numeric range dominate the result. Fusion is therefore a calibration problem as much as a recall problem.

Mental model

Fusion is a committee vote where each member ranks the candidates. A candidate ranked highly by several members is likely strong, even if no single member put it first. The metaphor breaks when the members are correlated: if both retrievers learned the same bias, agreement adds confidence without adding information.

How it works

Mechanism, step by stepInteractive
  1. 1Run each retriever independently and keep its ranked list with stable document identifiers.
  2. 2Deduplicate across lists by document id or content hash before combining, preserving the best rank per retriever.
  3. 3Choose rank-based fusion such as reciprocal rank fusion when score scales differ or are unknown.
  4. 4If fusing scores, normalise per list first, using min-max or z-score within the returned set, and document the choice.
  5. 5Tune the fusion constant or weights on a judged set, and check that gains hold on identifier and paraphrase slices separately.
  6. 6Cap each retriever's contribution so one slow or degraded source cannot flood the fused list.

Step 1 of 6

Run each retriever independently and keep its ranked list with stable document identifiers.

Step 1/6 · step 1

Figure. The mechanism in this page's own steps. The figure adds sequence, not new claims.

What to notice. A step can be read alone, but the order is the mechanism. The control lets you stop on any one.

Text version

The page's steps: 1. Run each retriever independently and keep its ranked list with stable document identifier… 2. Deduplicate across lists by document id or content hash before combining, preserving the … 3. Choose rank-based fusion such as reciprocal rank fusion when score scales differ or are u… 4. If fusing scores, normalise per list first, using min-max or z-score within the returned … 5. Tune the fusion constant or weights on a judged set, and check that gains hold on identif… 6. Cap each retriever's contribution so one slow or degraded source cannot flood the fused l…

  1. Run each retriever independently and keep its ranked list with stable document identifiers.
  2. Deduplicate across lists by document id or content hash before combining, preserving the best rank per retriever.
  3. Choose rank-based fusion such as reciprocal rank fusion when score scales differ or are unknown.
  4. If fusing scores, normalise per list first, using min-max or z-score within the returned set, and document the choice.
  5. Tune the fusion constant or weights on a judged set, and check that gains hold on identifier and paraphrase slices separately.
  6. Cap each retriever's contribution so one slow or degraded source cannot flood the fused list.

Trade-offs

ChoiceGood forCostFails when
Reciprocal rank fusionNo score calibration, robust across enginesIgnores score magnitude and confidence gapsOne retriever is much stronger than the other
Weighted score fusionExploits calibrated confidenceRequires normalisation and tuning per corpusScore distributions shift between queries
Fixed equal weightsSimple and stableCannot adapt when one retriever degradesTraffic mixes query types with different winners
Fuse then rerankBest precision at the topTwo model stages and higher latencyLatency budget is already tight
Trade-off mapInteractive

Good for

No score calibration, robust across engines

Cost

Ignores score magnitude and confidence gaps

Fails when

One retriever is much stronger than the other

  1. One retriever is much stronger than the other
  2. Score distributions shift between queries
  3. Traffic mixes query types with different winners
  4. Latency budget is already tight

Figure. Each choice with what it buys, what it costs, and the case where it loses.

What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.

Text version

Four trade-offs: Reciprocal rank fusion is good for No score calibration, robust across engines, costs Ignores score magnitude and confidence gaps, and fails when One retriever is much stronger than the other. Weighted score fusion is good for Exploits calibrated confidence, costs Requires normalisation and tuning per corpus, and fails when Score distributions shift between queries. Fixed equal weights is good for Simple and stable, costs Cannot adapt when one retriever degrades, and fails when Traffic mixes query types with different winners. Fuse then rerank is good for Best precision at the top, costs Two model stages and higher latency, and fails when Latency budget is already tight

Failure modes

  • Unnormalised score addition. BM25 scores in the tens overwhelm cosine similarities below one, so the dense list effectively disappears. Debug: print the score ranges of each list for one query and check whether one range dominates.
  • Duplicate documents across lists. The same passage appears twice under different ids and consumes two context slots. Debug: group the fused list by content hash and count collisions.
  • Correlated retrievers. Both lists return the same wrong passage, and fusion promotes it. Debug: compute the overlap of the two lists; high overlap with low answer quality means fusion cannot help.

In production

Track per-retriever contribution to the final top-k and the overlap between lists. If one source never contributes, it is costing latency without buying recall.

Common misconceptions

People think: Fusion always beats the best single retriever.

Actually: It beats each retriever only on queries where their failures differ; on a homogeneous failure it just adds cost.

People think: Scores from two engines can be added directly.

Actually: Unless both are calibrated to a common scale, the sum is dominated by whichever produces larger numbers.

Interview angle

Contrast rank-based and score-based fusion, and explain why calibration is the sticking point. Describe the overlap test that tells you whether fusion can help at all. Averaging two score lists without normalisation, or ignoring duplicate documents across lists, is where most answers fall apart.

Sources