Skip to content
AI Engineer Guide.
seniorReportedcurated

IVF, HNSW, product quantisation — pick an index for 500M vectors under a hard memory budget and defend it.

30-second answer

Likely follow-ups

Open a question to rehearse the answer.

How do you measure the recall you gave up against exact search, and what target is acceptable?
  • Run exact search on a sample restricted to the same filters and compare the compressed index's top-k against that ground truth.
  • Set an acceptance target before the migration and block the rollout when compressed recall misses it on the hard slice.
  • Compare exact and compressed rankings for queries whose answers are known, since compression error is uneven across regions of the space.
How does a high insert or delete rate change the IVF versus HNSW choice?
  • HNSW pays expensive deletes and graph degradation under churn, so a high delete rate argues for periodic rebuilds or an IVF-family index.
  • IVF needs its coarse quantiser retrained as the distribution shifts, so heavy inserts also cost rebuild cycles.
  • Batch writes and rebuild off-peak whichever family you choose, because churn shows up as recall drift rather than an error.
Where does exact rescoring fit, and how many candidates must you rescore to recover the loss?
  • Rescore the top candidates with full-precision vectors, which means keeping a copy of them somewhere cheaper than the main index.
  • Derive the depth from measurement: raise the rescore count until compressed recall against ground truth meets the target.
  • Rescoring recovers ordering, not missing candidates, so a passage the compressed search never returned cannot be rescued.

Red flags

  • PQ as lossless
  • cannot explain nprobe

Concepts this tests

Tags

Source: ishwarj.com

More in RAG & retrieval pipelines