seniorReportedcurated
IVF, HNSW, product quantisation — pick an index for 500M vectors under a hard memory budget and defend it.
30-second answer
- The three are orthogonal mechanisms: IVF prunes by cluster, HNSW prunes by graph links, and PQ compresses vectors into short codes.
- IVF keeps every full vector and saves only the graph's link storage, so it is lighter than HNSW but compresses nothing, and cluster skew plus nprobe still decide recall.
- HNSW gives the best recall per millisecond at moderate scale but keeps vectors and graph in RAM, which is the binding constraint at 500 million.
- PQ compresses sub-vectors to centroid IDs, 8-64 times smaller and lossy, usually paired as IVF-PQ or HNSW-PQ with exact rescoring.
- At this scale under memory pressure I would ship IVF-PQ or binary-quantised HNSW plus rescoring, and state the recall hit explicitly rather than claiming parity.
Likely follow-ups
Open a question to rehearse the answer.
How do you measure the recall you gave up against exact search, and what target is acceptable?
- Run exact search on a sample restricted to the same filters and compare the compressed index's top-k against that ground truth.
- Set an acceptance target before the migration and block the rollout when compressed recall misses it on the hard slice.
- Compare exact and compressed rankings for queries whose answers are known, since compression error is uneven across regions of the space.
How does a high insert or delete rate change the IVF versus HNSW choice?
- HNSW pays expensive deletes and graph degradation under churn, so a high delete rate argues for periodic rebuilds or an IVF-family index.
- IVF needs its coarse quantiser retrained as the distribution shifts, so heavy inserts also cost rebuild cycles.
- Batch writes and rebuild off-peak whichever family you choose, because churn shows up as recall drift rather than an error.
Where does exact rescoring fit, and how many candidates must you rescore to recover the loss?
- Rescore the top candidates with full-precision vectors, which means keeping a copy of them somewhere cheaper than the main index.
- Derive the depth from measurement: raise the rescore count until compressed recall against ground truth meets the target.
- Rescoring recovers ordering, not missing candidates, so a passage the compressed search never returned cannot be rescued.
Red flags
- PQ as lossless
- cannot explain nprobe
Concepts this tests
Tags
Source: ishwarj.com
More in RAG & retrieval pipelines
- Explain parent-child (small-to-big) retrieval. What problem does it solve and what does it cost you?
- Explain HNSW. What do M, ef_construction and ef_search each control, and which can you change in production?
- How would you tune HNSW for 10 million vectors when the product needs p95 retrieval under 100 ms?