Why it matters
Retrieval quality depends on the index and on how filters interact with it. Two systems running the same HNSW parameters can return different results when a metadata filter is applied: one pre-filters candidates, the other filters after the approximate search and silently returns fewer than k results. That difference is invisible in a demo and decisive in production.
Mental model
A vector index is a library catalogue organised by similarity instead of alphabet. Filters are the librarian's instructions about which shelves to search. The metaphor breaks when the librarian guesses: approximate search may visit only some shelves, so a filter applied afterwards can return an empty stack.
How it works
- 1Estimate corpus size, vector dimension and growth to size memory and index build time.
- 2List the filters your product needs and check whether the engine pre-filters or post-filters.
- 3Choose an index: HNSW for low-latency in-memory, IVF for memory-constrained, disk-based for very large corpora.
- 4Test recall against exact search on a sample, with filters enabled.
- 5Plan ingestion, updates and deletes; approximate indexes degrade with churn.
- 6Check operational requirements: backups, replicas, access control, observability and managed options.
Step 1 of 6
Estimate corpus size, vector dimension and growth to size memory and index build time.
Figure. The mechanism in this page's own steps. The figure adds sequence, not new claims.
What to notice. A step can be read alone, but the order is the mechanism. The control lets you stop on any one.
Text version
The page's steps: 1. Estimate corpus size, vector dimension and growth to size memory and index build time. 2. List the filters your product needs and check whether the engine pre-filters or post-filt… 3. Choose an index: HNSW for low-latency in-memory, IVF for memory-constrained, disk-based f… 4. Test recall against exact search on a sample, with filters enabled. 5. Plan ingestion, updates and deletes; approximate indexes degrade with churn. 6. Check operational requirements: backups, replicas, access control, observability and mana…
- Estimate corpus size, vector dimension and growth to size memory and index build time.
- List the filters your product needs and check whether the engine pre-filters or post-filters.
- Choose an index: HNSW for low-latency in-memory, IVF for memory-constrained, disk-based for very large corpora.
- Test recall against exact search on a sample, with filters enabled.
- Plan ingestion, updates and deletes; approximate indexes degrade with churn.
- Check operational requirements: backups, replicas, access control, observability and managed options.
Trade-offs
| Choice | Good for | Cost | Fails when |
|---|---|---|---|
| Postgres with pgvector | One system, transactions, SQL filters | Recall and scale ceiling at high concurrency | Billion-scale vectors or heavy filtering |
| Dedicated vector database | Tuned indexes and filtering | Another service to operate | Corpus fits comfortably in Postgres |
| In-memory HNSW | Low latency, simple operations | Memory cost, slow cold start | Corpus exceeds RAM |
| Disk-based index | Large corpora at lower cost | Higher query latency | Interactive latency budget is tight |
Good for
One system, transactions, SQL filters
Cost
Recall and scale ceiling at high concurrency
Fails when
Billion-scale vectors or heavy filtering
- Billion-scale vectors or heavy filtering
- Corpus fits comfortably in Postgres
- Corpus exceeds RAM
- Interactive latency budget is tight
Figure. Each choice with what it buys, what it costs, and the case where it loses.
What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.
Text version
Four trade-offs: Postgres with pgvector is good for One system, transactions, SQL filters, costs Recall and scale ceiling at high concurrency, and fails when Billion-scale vectors or heavy filtering. Dedicated vector database is good for Tuned indexes and filtering, costs Another service to operate, and fails when Corpus fits comfortably in Postgres. In-memory HNSW is good for Low latency, simple operations, costs Memory cost, slow cold start, and fails when Corpus exceeds RAM. Disk-based index is good for Large corpora at lower cost, costs Higher query latency, and fails when Interactive latency budget is tight
Failure modes
- Post-filter truncation. A filtered query returns fewer than k results because filtering happens after the search. Debug: compare result counts with and without filters.
- Recall measured without filters. Benchmarks pass, production queries miss. Debug: include the real filter distribution in the recall test.
- Index churn. Frequent updates degrade graph quality and latency. Debug: monitor rebuild frequency and search recall over time.
In production
Instrument query latency, recall against a sampled exact search, and index size. Keep a rebuild path and a tested restore. Start with the operational database and move only when a measured limit is hit.
Common misconceptions
People think: A dedicated vector database is required for RAG.
Actually: Most products start well under a million vectors, where an existing database with a vector extension is simpler and correct.
People think: Approximate indexes are close enough to exact search.
Actually: Recall depends on parameters, data distribution and filters; it must be measured, not assumed.
Interview angle
Compare on index behaviour, filtering semantics, scale and operations, and justify starting simple. Name the recall test and the post-filter trap. A weak answer lists products; a strong one names the constraint that would force a migration.
Sources
- pgvector — index types, operators and filtering in Postgres.
- Malkov & Yashunin, HNSW — the graph index behind most low-latency systems.
- Qdrant documentation — filtering semantics and payload indexing in a dedicated engine.
Related
- Serving engines compared
- Embedding models compared