seniorPracticecurated
What does the retriever do when a query has no relevant documents at all? Design the abstain path.
30-second answer
- A retriever always returns its top-k, so no-answer has to be an explicit decision rather than something the index can express.
- Calibrate a threshold on labelled answerable and unanswerable questions, and score it with a reranker score or a small classifier, because raw cosine scores are not comparable across queries.
- If nothing clears the threshold the system abstains, says what it could not find, and offers a human path instead of answering from parametric memory.
- When the product promise is grounded citations, an answer from memory without evidence is a failure rather than a graceful fallback.
- A conservative threshold protects trust and increases deflection to humans, so the setting is a product decision about which error costs more.
Likely follow-ups
Open a question to rehearse the answer.
How do you calibrate an abstain threshold when you have no labelled unanswerable questions?
- Mine candidates from production: queries whose best candidate scored low, and questions users rephrased after receiving an answer.
- Generate unanswerable variants about entities or periods absent from the corpus, then hand-check a sample to confirm genuine absence.
- Start permissive enough that false abstentions on answerable questions are measured, then tighten until misses fall, recording the miss versus over-block trade-off per category.
Should the system answer partially when one of three sub-claims is unsupported?
- Answer the supported parts and state which part could not be supported, because a silent omission reads to the user as a complete answer.
- Attribute evidence per sub-question so synthesis detects the gap instead of merging what it happens to have.
- Abstain entirely when the missing part changes the decision, since a partial answer to a comparison misleads more than a refusal.
How do you tell a genuine no-answer from a retrieval miss, and does the user see the difference?
- A genuine no-answer has no supporting passage in the corpus; a miss has one that search failed to surface, so run exact search before concluding.
- Treat a miss as a retrieval bug with a fix and a genuine no-answer as an abstention with a product response.
- Internally the two carry different reason codes, while the user sees one honest message because the distinction is not actionable for them.
Red flags
- always answers
- threshold by vibes
Concepts this tests
Tags
More in RAG & retrieval pipelines
- Explain parent-child (small-to-big) retrieval. What problem does it solve and what does it cost you?
- Explain HNSW. What do M, ef_construction and ef_search each control, and which can you change in production?
- How would you tune HNSW for 10 million vectors when the product needs p95 retrieval under 100 ms?