Skip to content
AI Engineer Guide.

concept · concepts · intermediate · concept 36 of 57

MMR

0 of 57 complete

Why it matters

Top-k by similarity often returns several passages that say the same thing, because near-duplicates cluster at the top of a vector index. The prompt then spends its budget on one fact and omits the rest. MMR treats the context as a set to be filled, penalising a candidate that resembles something already chosen, which raises the amount of distinct evidence per token.

Mental model

MMR is packing a suitcase: after the first useful item, you prefer something different over a second copy of it. The metaphor breaks when the trip needs two of the same thing — a question that genuinely requires two passages on one topic gets penalised for the second.

How it works

Mechanism, step by stepInteractive
  1. 1Retrieve a candidate pool wider than the final context, since diversity selection can only choose from what it has.
  2. 2Score each candidate for relevance to the query using the same retriever or a reranker.
  3. 3Pick the top candidate, then iteratively choose the one maximising lambda times relevance minus one minus lambda times maximum similarity to the selected set.
  4. 4Set lambda from the question type: high for factoid lookups, lower for broad or comparative questions.
  5. 5Recompute similarity against the selected set in the embedding space used for retrieval, and normalise scores to a comparable range.
  6. 6Stop at the token budget rather than a fixed count, and record why each passage was selected.

Step 1 of 6

Retrieve a candidate pool wider than the final context, since diversity selection can only choose from what it has.

Step 1/6 · step 1

Figure. The mechanism in this page's own steps. The figure adds sequence, not new claims.

What to notice. A step can be read alone, but the order is the mechanism. The control lets you stop on any one.

Text version

The page's steps: 1. Retrieve a candidate pool wider than the final context, since diversity selection can onl… 2. Score each candidate for relevance to the query using the same retriever or a reranker. 3. Pick the top candidate, then iteratively choose the one maximising lambda times relevance… 4. Set lambda from the question type: high for factoid lookups, lower for broad or comparati… 5. Recompute similarity against the selected set in the embedding space used for retrieval, … 6. Stop at the token budget rather than a fixed count, and record why each passage was selec…

  1. Retrieve a candidate pool wider than the final context, since diversity selection can only choose from what it has.
  2. Score each candidate for relevance to the query using the same retriever or a reranker.
  3. Pick the top candidate, then iteratively choose the one maximising lambda times relevance minus one minus lambda times maximum similarity to the selected set.
  4. Set lambda from the question type: high for factoid lookups, lower for broad or comparative questions.
  5. Recompute similarity against the selected set in the embedding space used for retrieval, and normalise scores to a comparable range.
  6. Stop at the token budget rather than a fixed count, and record why each passage was selected.

Trade-offs

ChoiceGood forCostFails when
High lambdaPrecision on single-fact questionsRedundant contextThe question needs several distinct sub-topics
Low lambdaCoverage across sub-topicsDrops the second-best evidence for a weaker but novel passageThe answer needs two passages on the same point
MMR after rerankingDiverse and preciseTwo scoring stagesLatency budget is tight
Plain top-kSimplest and fastestNear-duplicate passages fill the promptThe corpus repeats boilerplate
Trade-off mapInteractive

Good for

Precision on single-fact questions

Cost

Redundant context

Fails when

The question needs several distinct sub-topics

  1. The question needs several distinct sub-topics
  2. The answer needs two passages on the same point
  3. Latency budget is tight
  4. The corpus repeats boilerplate

Figure. Each choice with what it buys, what it costs, and the case where it loses.

What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.

Text version

Four trade-offs: High lambda is good for Precision on single-fact questions, costs Redundant context, and fails when The question needs several distinct sub-topics. Low lambda is good for Coverage across sub-topics, costs Drops the second-best evidence for a weaker but novel passage, and fails when The answer needs two passages on the same point. MMR after reranking is good for Diverse and precise, costs Two scoring stages, and fails when Latency budget is tight. Plain top-k is good for Simplest and fastest, costs Near-duplicate passages fill the prompt, and fails when The corpus repeats boilerplate

Failure modes

  • Diversity at the cost of the answer. The correct passage is dropped for a novel but irrelevant one. Debug: check whether the gold passage was in the candidate pool and what its similarity to the selected set was.
  • Similarity measured across incomparable scores. Relevance from BM25 is subtracted from a cosine similarity and the units do not match. Debug: print both terms for a selection step and check their ranges.
  • Lambda tuned on one question type. Factoid queries improve while comparative ones regress. Debug: evaluate per question type with lambda held fixed.

In production

Log the redundancy of the final context — mean pairwise similarity — and the share of questions where the gold passage was dropped by diversity. Cap the candidate pool so selection stays cheap.

Common misconceptions

People think: MMR improves recall.

Actually: It only chooses among retrieved candidates; anything the first stage missed stays missing.

People think: More diversity is always better.

Actually: Diversity competes with relevance, and a question that needs two passages on one fact is harmed by it.

Interview angle

State the greedy objective, explain lambda as a dial, and separate redundancy from recall. Name the question type where diversity hurts. Saying MMR removes duplicates, with no notion of the relevance penalty, is only half the mechanism.

Sources