Why it matters
Product requirements turn retrieval into a filtered problem: only this customer's documents, only the current policy version, only the region that applies. An approximate index built over the whole corpus knows nothing about those constraints, so the engine must either restrict the search space or discard results afterwards. The second option looks simpler and fails exactly when the filter is most necessary, because the top candidates are mostly ineligible.
Mental model
Filtering is asking a librarian for books from one shelf. Pre-filtering walks to that shelf; post-filtering grabs the ten nearest books in the building and throws away the ones from other shelves. The metaphor breaks because the librarian cannot tell you the shelf was nearly empty until you count what is left in your hands.
How it works
- 1List every filter the product needs and classify it as tenant, permission, version, time or attribute.
- 2Check the engine's filtering semantics: does it restrict the graph walk, or filter after the candidate set is formed?
- 3Map low-cardinality filters to partitions or separate collections so the search space shrinks before the walk.
- 4For selective filters, over-fetch by a factor and assert that k eligible results are returned.
- 5Store filter values in typed columns rather than free text, so range and equality predicates are indexable.
- 6Test with the real filter distribution, including the narrowest tenant, not only with a filter that matches everything.
Step 1 of 6
List every filter the product needs and classify it as tenant, permission, version, time or attribute.
Figure. The mechanism in this page's own steps. The figure adds sequence, not new claims.
What to notice. A step can be read alone, but the order is the mechanism. The control lets you stop on any one.
Text version
The page's steps: 1. List every filter the product needs and classify it as tenant, permission, version, time … 2. Check the engine's filtering semantics: does it restrict the graph walk, or filter after … 3. Map low-cardinality filters to partitions or separate collections so the search space shr… 4. For selective filters, over-fetch by a factor and assert that k eligible results are retu… 5. Store filter values in typed columns rather than free text, so range and equality predica… 6. Test with the real filter distribution, including the narrowest tenant, not only with a f…
- List every filter the product needs and classify it as tenant, permission, version, time or attribute.
- Check the engine's filtering semantics: does it restrict the graph walk, or filter after the candidate set is formed?
- Map low-cardinality filters to partitions or separate collections so the search space shrinks before the walk.
- For selective filters, over-fetch by a factor and assert that k eligible results are returned.
- Store filter values in typed columns rather than free text, so range and equality predicates are indexable.
- Test with the real filter distribution, including the narrowest tenant, not only with a filter that matches everything.
Trade-offs
| Choice | Good for | Cost | Fails when |
|---|---|---|---|
| Pre-filtering | Correct result counts under selective filters | Engine support varies; may scan more than a graph walk | The filter is broad and the index is graph-based |
| Post-filtering with over-fetch | Works on any engine | Wasted candidates and unpredictable counts | Filters are selective or correlated with distance |
| Partition per tenant | Small search spaces and clean isolation | Many partitions and cross-tenant queries become awkward | Tenants are numerous and tiny |
| Filter as a soft boost | Never returns empty | Leaks ineligible documents | The filter is a permission boundary |
Good for
Correct result counts under selective filters
Cost
Engine support varies; may scan more than a graph walk
Fails when
The filter is broad and the index is graph-based
- The filter is broad and the index is graph-based
- Filters are selective or correlated with distance
- Tenants are numerous and tiny
- The filter is a permission boundary
Figure. Each choice with what it buys, what it costs, and the case where it loses.
What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.
Text version
Four trade-offs: Pre-filtering is good for Correct result counts under selective filters, costs Engine support varies; may scan more than a graph walk, and fails when The filter is broad and the index is graph-based. Post-filtering with over-fetch is good for Works on any engine, costs Wasted candidates and unpredictable counts, and fails when Filters are selective or correlated with distance. Partition per tenant is good for Small search spaces and clean isolation, costs Many partitions and cross-tenant queries become awkward, and fails when Tenants are numerous and tiny. Filter as a soft boost is good for Never returns empty, costs Leaks ineligible documents, and fails when The filter is a permission boundary
Failure modes
- Fewer results than requested. A selective filter leaves three candidates out of a requested ten, and the prompt is under-filled. Debug: log candidate counts before and after filtering for the narrowest filter in use.
- Permission filter applied after ranking. An ineligible document is ranked, then dropped, and its presence still influenced the ordering or the cached answer. Debug: assert that the filter predicate is part of the query, then verify no ineligible id appears in the raw candidate list.
- Filter value drift. Documents are indexed with an old tenant identifier and become invisible after a rename. Debug: count documents per filter value and compare with the source system.
In production
Monitor the distribution of eligible-result counts per filter and alert when it approaches zero. Keep filters inside the query plan, and rehearse the narrowest tenant on every index change.
Common misconceptions
People think: Filtering after the search is equivalent to filtering before it.
Actually: It changes both the result count and which documents are considered, so recall under selective filters differs.
People think: Metadata is just for display.
Actually: Filter fields are part of the index contract and determine which documents a query can reach.
Interview angle
Ask first whether the engine pre-filters, then design partitions and over-fetch around the answer. Treat permission filters as a correctness boundary, not a ranking preference. The weak version adds a metadata filter and never checks whether k results came back.
Sources
- pgvector — how filtering interacts with HNSW and IVFFlat scans.
- Qdrant, Filtering — filterable fields and payload conditions in a vector engine.
- Elasticsearch, k-nearest neighbour search — filtered kNN and the candidate-count parameter that controls it.