Skip to content
AI Engineer Guide.
midReportedcurated

The vector database already returns similarity scores. Why add a reranker, and how deep do you rerank?

30-second answer

Likely follow-ups

Open a question to rehearse the answer.

How do you measure the reranker's lift without confounding it with retrieval changes?
  • Hold retrieval fixed and score the same candidate lists with and without the reranker, comparing nDCG@5 and answer correctness.
  • Record candidate IDs and rerank scores per request, so a regression can be replayed instead of guessed.
  • If ranking improves and answers do not, the bottleneck is assembly or generation rather than the reranker.
How does a late-interaction model like ColBERT compare on storage and latency, and when would you pick it?
  • Late interaction keeps a vector per token, so the index is far larger than a single-vector index and memory becomes the constraint.
  • It scores with cheap per-token interactions at query time, landing between bi-encoder speed and cross-encoder accuracy.
  • Pick it when you need finer matching than a pooled vector but cannot afford a forward pass per candidate.
Your reranker improves nDCG but end-to-end correctness is flat, what do you conclude?
  • The ranking metric is not the binding constraint: the evidence was already in the prompt and the generator is losing it.
  • Check assembly next, since a correct passage buried in the middle of a long context gets ignored even when ranked first.
  • Then check the labels, because a judged ranking that does not reflect answerability can improve while answers do not.

Red flags

  • vague "understands context"
  • reranks thousands

Concepts this tests

Tags

Source: ishwarj.com

More in RAG & retrieval pipelines