staffReportedcurated
You have 25M chunks at 1,536 dimensions. Size the index and tell me how you make it fit — then what changes at 10x.
30-second answer
- The raw vectors are 25 million times 1,536 times 4 bytes, about 150 GB, and HNSW links add roughly 5 to 10 percent, so the resident footprint is around 160 GB before any replication.
- The levers in order are int8 scalar quantisation at about 4x, Matryoshka truncation to 512 or 768 dimensions at another 2 to 3x, then sharding, with quantised candidates rescored against originals kept on disk.
- Int8 alone brings the vectors to about 38 GB, and int8 at 768 dimensions is about 19 GB, which fits a single large node before link overhead.
- The trade-off is recall and operational complexity against RAM cost, and compression changes distance estimates near the decision boundary rather than only storage.
- At 10x, recall at a fixed ef also degrades, so re-tune ef_search against brute-force ground truth instead of assuming the same settings transfer.
Likely follow-ups
Open a question to rehearse the answer.
How do you measure the recall you lose to quantisation before committing?
- Take a few hundred labelled queries, compute exact top-k by brute force on the float vectors, and compare the quantised index's recall@k at the ef you plan to run.
- Sweep ef_search and plot recall against latency, because the honest question is what recall costs at the latency budget, not the loss at one setting.
- Rescore the top candidates against full-precision vectors and measure the recovered recall, since that is the configuration you would actually ship.
At 10x, how do you shard without wrecking recall for users whose filters touch every shard?
- Shard by tenant or document partition when filters are predictable, so a query touches one shard and the filter does not starve the graph walk.
- For filters that span every shard, scatter to all shards, retrieve a deeper local top-k, and merge globally; the cost is latency and fan-out, not recall.
- Keep a routing table for tenant to shard and test a foreign-tenant query for an empty result, because cross-shard leakage is a filter bug rather than an index bug.
Walk me through the arithmetic for int8 plus Matryoshka truncation to 768 dimensions — what stays resident then?
- 25 million vectors at 768 int8 components is 25 million times 768 bytes, about 19.2 GB, before links.
- Add 5 to 10 percent for HNSW links, so budget roughly 20 to 21 GB resident for the graph and vectors.
- That leaves activations, the reranking originals on disk, and replication outside the number, so the node size must cover those separately.
Red flags
- guesses with no arithmetic
- says quantisation is free
Concepts this tests
Tags
Source: qdrant.tech
More in Embeddings & retrieval
- Where do embeddings systematically fail as a retrieval mechanism, and which failures can you fix cheaply?
- You must choose an embedding model for a 900k-chunk support corpus. Walk me through the bake-off and the constraints that don't appear on a leaderboard.
- Why does swapping the embedding model force a full re-index, and how do you roll that out without downtime?