Skip to content
AI Engineer Guide.
seniorReportedcurated

Weighted min-max fusion with an alpha versus reciprocal rank fusion — which do you ship, and how do you tune it?

30-second answer

Likely follow-ups

Open a question to rehearse the answer.

When would distribution-based score fusion beat RRF, and what do you need to calibrate it?
  • It can win when each retriever's score scale is stable enough that normalising per list makes the magnitudes comparable, so a confident retriever can outrank agreement.
  • The normalisation happens per list at query time, so fusing needs no labels; a judged set is only there to check the gain holds per query type.
  • It drifts when a model upgrade moves the score scale, so the calibration is a maintained artefact rather than a one-off setting.
How do you join the two candidate lists, by ID or by position, and what goes wrong with the other?
  • Join by stable document ID and dedupe on content hash, since the same passage often arrives from both lists under different IDs after re-chunking.
  • Joining by position pairs unrelated documents, because rank three in one list has nothing to do with rank three in the other.
  • Keep the best rank per retriever per document, so agreement is counted once instead of inflating a document that appears twice.
How many labelled queries per question type before an alpha sweep means anything?
  • Size each stratum from the interval you need, so the gap between two alphas is larger than the uncertainty on that slice.
  • Stratify by query type first and size each stratum separately, because identifier and paraphrase traffic respond differently to the same weight.
  • Report the per-slice distribution and the worst slice rather than the mean, since the aggregate hides the query type the weight broke.
  • Hold out part of each stratum, because an alpha tuned on the queries it is scored on will not transfer.

Red flags

  • add raw scores
  • tune on five queries

Concepts this tests

Tags

Source: forwarddeployedprep.com

More in RAG & retrieval pipelines