Skip to content
AI Engineer Guide.
staffReportedcurated

How would you tune HNSW for 10 million vectors when the product needs p95 retrieval under 100 ms?

30-second answer

Likely follow-ups

Open a question to rehearse the answer.

Would you quantise before or after sweeping ef_search, and how does that order change the answer?
  • Sweep at full precision first, so the recall loss from compression is measured against a known baseline instead of being confounded with ef.
  • Then re-sweep ef_search on the quantised index, because the same ef yields lower recall and the old value is stale.
  • Keep exact rescoring available and measure how many candidates must be rescored to recover the target.
How do you keep the brute-force ground-truth sample from drifting as the corpus grows?
  • Freeze a query sample with its exact top-k, then re-run exact search against the current corpus on a schedule and diff.
  • Refresh the query sample from recent logs, or the ground truth measures questions the product no longer receives.
  • Alert when exact and ANN disagree beyond the target, which catches both index drift and corpus change.
p95 meets the target but p99 does not, what do you change, and what does it cost in recall?
  • Raise ef_search to lift the tail, and check whether the cost in p50 latency is acceptable on the same traffic.
  • Look for queueing and cold replicas before touching the index, since the tail often comes from admission rather than search.
  • If the tail remains, cap query-time effort and over-fetch so one expensive query cannot consume the budget.

Red flags

  • tutorial defaults
  • optimizes mean only

Concepts this tests

Tags

Source: ishwarj.com

More in RAG & retrieval pipelines