seniorReportedcurated
Weighted min-max fusion with an alpha versus reciprocal rank fusion — which do you ship, and how do you tune it?
30-second answer
- Ship RRF unless you have evidence that a tuned blend beats it, because min-max normalisation is relative to each query's own result list.
- With min-max the top chunk scores 1.0 even when nothing matches, and one outlier compresses everything else, so alpha needs retuning per corpus and query class.
- RRF uses ranks only, summing 1/(k + rank) with k near 60, needs no calibration, and rewards chunks both retrievers agree on.
- Tune weights on the golden set split by question type rather than on the mean, because the aggregate hides the slice you broke.
- RRF discards magnitude, so a confidently correct retriever is not rewarded, and high overlap between the lists means fusion adds cost without recall.
Likely follow-ups
Open a question to rehearse the answer.
When would distribution-based score fusion beat RRF, and what do you need to calibrate it?
- It can win when each retriever's score scale is stable enough that normalising per list makes the magnitudes comparable, so a confident retriever can outrank agreement.
- The normalisation happens per list at query time, so fusing needs no labels; a judged set is only there to check the gain holds per query type.
- It drifts when a model upgrade moves the score scale, so the calibration is a maintained artefact rather than a one-off setting.
How do you join the two candidate lists, by ID or by position, and what goes wrong with the other?
- Join by stable document ID and dedupe on content hash, since the same passage often arrives from both lists under different IDs after re-chunking.
- Joining by position pairs unrelated documents, because rank three in one list has nothing to do with rank three in the other.
- Keep the best rank per retriever per document, so agreement is counted once instead of inflating a document that appears twice.
How many labelled queries per question type before an alpha sweep means anything?
- Size each stratum from the interval you need, so the gap between two alphas is larger than the uncertainty on that slice.
- Stratify by query type first and size each stratum separately, because identifier and paraphrase traffic respond differently to the same weight.
- Report the per-slice distribution and the worst slice rather than the mean, since the aggregate hides the query type the weight broke.
- Hold out part of each stratum, because an alpha tuned on the queries it is scored on will not transfer.
Red flags
- add raw scores
- tune on five queries
Concepts this tests
Tags
Source: forwarddeployedprep.com
More in RAG & retrieval pipelines
- Explain parent-child (small-to-big) retrieval. What problem does it solve and what does it cost you?
- Explain HNSW. What do M, ef_construction and ef_search each control, and which can you change in production?
- How would you tune HNSW for 10 million vectors when the product needs p95 retrieval under 100 ms?