Skip to content
AI Engineer Guide.

concept · data · intermediate · concept 1 of 3

Vector DBs compared

0 of 3 complete

Why it matters

Retrieval quality depends on the index and on how filters interact with it. Two systems running the same HNSW parameters can return different results when a metadata filter is applied: one pre-filters candidates, the other filters after the approximate search and silently returns fewer than k results. That difference is invisible in a demo and decisive in production.

Mental model

A vector index is a library catalogue organised by similarity instead of alphabet. Filters are the librarian's instructions about which shelves to search. The metaphor breaks when the librarian guesses: approximate search may visit only some shelves, so a filter applied afterwards can return an empty stack.

How it works

Mechanism, step by stepInteractive
  1. 1Estimate corpus size, vector dimension and growth to size memory and index build time.
  2. 2List the filters your product needs and check whether the engine pre-filters or post-filters.
  3. 3Choose an index: HNSW for low-latency in-memory, IVF for memory-constrained, disk-based for very large corpora.
  4. 4Test recall against exact search on a sample, with filters enabled.
  5. 5Plan ingestion, updates and deletes; approximate indexes degrade with churn.
  6. 6Check operational requirements: backups, replicas, access control, observability and managed options.

Step 1 of 6

Estimate corpus size, vector dimension and growth to size memory and index build time.

Step 1/6 · step 1

Figure. The mechanism in this page's own steps. The figure adds sequence, not new claims.

What to notice. A step can be read alone, but the order is the mechanism. The control lets you stop on any one.

Text version

The page's steps: 1. Estimate corpus size, vector dimension and growth to size memory and index build time. 2. List the filters your product needs and check whether the engine pre-filters or post-filt… 3. Choose an index: HNSW for low-latency in-memory, IVF for memory-constrained, disk-based f… 4. Test recall against exact search on a sample, with filters enabled. 5. Plan ingestion, updates and deletes; approximate indexes degrade with churn. 6. Check operational requirements: backups, replicas, access control, observability and mana…

  1. Estimate corpus size, vector dimension and growth to size memory and index build time.
  2. List the filters your product needs and check whether the engine pre-filters or post-filters.
  3. Choose an index: HNSW for low-latency in-memory, IVF for memory-constrained, disk-based for very large corpora.
  4. Test recall against exact search on a sample, with filters enabled.
  5. Plan ingestion, updates and deletes; approximate indexes degrade with churn.
  6. Check operational requirements: backups, replicas, access control, observability and managed options.

Trade-offs

ChoiceGood forCostFails when
Postgres with pgvectorOne system, transactions, SQL filtersRecall and scale ceiling at high concurrencyBillion-scale vectors or heavy filtering
Dedicated vector databaseTuned indexes and filteringAnother service to operateCorpus fits comfortably in Postgres
In-memory HNSWLow latency, simple operationsMemory cost, slow cold startCorpus exceeds RAM
Disk-based indexLarge corpora at lower costHigher query latencyInteractive latency budget is tight
Trade-off mapInteractive

Good for

One system, transactions, SQL filters

Cost

Recall and scale ceiling at high concurrency

Fails when

Billion-scale vectors or heavy filtering

  1. Billion-scale vectors or heavy filtering
  2. Corpus fits comfortably in Postgres
  3. Corpus exceeds RAM
  4. Interactive latency budget is tight

Figure. Each choice with what it buys, what it costs, and the case where it loses.

What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.

Text version

Four trade-offs: Postgres with pgvector is good for One system, transactions, SQL filters, costs Recall and scale ceiling at high concurrency, and fails when Billion-scale vectors or heavy filtering. Dedicated vector database is good for Tuned indexes and filtering, costs Another service to operate, and fails when Corpus fits comfortably in Postgres. In-memory HNSW is good for Low latency, simple operations, costs Memory cost, slow cold start, and fails when Corpus exceeds RAM. Disk-based index is good for Large corpora at lower cost, costs Higher query latency, and fails when Interactive latency budget is tight

Failure modes

  • Post-filter truncation. A filtered query returns fewer than k results because filtering happens after the search. Debug: compare result counts with and without filters.
  • Recall measured without filters. Benchmarks pass, production queries miss. Debug: include the real filter distribution in the recall test.
  • Index churn. Frequent updates degrade graph quality and latency. Debug: monitor rebuild frequency and search recall over time.

In production

Instrument query latency, recall against a sampled exact search, and index size. Keep a rebuild path and a tested restore. Start with the operational database and move only when a measured limit is hit.

Common misconceptions

People think: A dedicated vector database is required for RAG.

Actually: Most products start well under a million vectors, where an existing database with a vector extension is simpler and correct.

People think: Approximate indexes are close enough to exact search.

Actually: Recall depends on parameters, data distribution and filters; it must be measured, not assumed.

Interview angle

Compare on index behaviour, filtering semantics, scale and operations, and justify starting simple. Name the recall test and the post-filter trap. A weak answer lists products; a strong one names the constraint that would force a migration.

Sources