Skip to content
AI Engineer Guide.

concept · concepts · intermediate · concept 14 of 57

Metadata filtering

0 of 57 complete

Why it matters

Product requirements turn retrieval into a filtered problem: only this customer's documents, only the current policy version, only the region that applies. An approximate index built over the whole corpus knows nothing about those constraints, so the engine must either restrict the search space or discard results afterwards. The second option looks simpler and fails exactly when the filter is most necessary, because the top candidates are mostly ineligible.

Mental model

Filtering is asking a librarian for books from one shelf. Pre-filtering walks to that shelf; post-filtering grabs the ten nearest books in the building and throws away the ones from other shelves. The metaphor breaks because the librarian cannot tell you the shelf was nearly empty until you count what is left in your hands.

How it works

Mechanism, step by stepInteractive
  1. 1List every filter the product needs and classify it as tenant, permission, version, time or attribute.
  2. 2Check the engine's filtering semantics: does it restrict the graph walk, or filter after the candidate set is formed?
  3. 3Map low-cardinality filters to partitions or separate collections so the search space shrinks before the walk.
  4. 4For selective filters, over-fetch by a factor and assert that k eligible results are returned.
  5. 5Store filter values in typed columns rather than free text, so range and equality predicates are indexable.
  6. 6Test with the real filter distribution, including the narrowest tenant, not only with a filter that matches everything.

Step 1 of 6

List every filter the product needs and classify it as tenant, permission, version, time or attribute.

Step 1/6 · step 1

Figure. The mechanism in this page's own steps. The figure adds sequence, not new claims.

What to notice. A step can be read alone, but the order is the mechanism. The control lets you stop on any one.

Text version

The page's steps: 1. List every filter the product needs and classify it as tenant, permission, version, time … 2. Check the engine's filtering semantics: does it restrict the graph walk, or filter after … 3. Map low-cardinality filters to partitions or separate collections so the search space shr… 4. For selective filters, over-fetch by a factor and assert that k eligible results are retu… 5. Store filter values in typed columns rather than free text, so range and equality predica… 6. Test with the real filter distribution, including the narrowest tenant, not only with a f…

  1. List every filter the product needs and classify it as tenant, permission, version, time or attribute.
  2. Check the engine's filtering semantics: does it restrict the graph walk, or filter after the candidate set is formed?
  3. Map low-cardinality filters to partitions or separate collections so the search space shrinks before the walk.
  4. For selective filters, over-fetch by a factor and assert that k eligible results are returned.
  5. Store filter values in typed columns rather than free text, so range and equality predicates are indexable.
  6. Test with the real filter distribution, including the narrowest tenant, not only with a filter that matches everything.

Trade-offs

ChoiceGood forCostFails when
Pre-filteringCorrect result counts under selective filtersEngine support varies; may scan more than a graph walkThe filter is broad and the index is graph-based
Post-filtering with over-fetchWorks on any engineWasted candidates and unpredictable countsFilters are selective or correlated with distance
Partition per tenantSmall search spaces and clean isolationMany partitions and cross-tenant queries become awkwardTenants are numerous and tiny
Filter as a soft boostNever returns emptyLeaks ineligible documentsThe filter is a permission boundary
Trade-off mapInteractive

Good for

Correct result counts under selective filters

Cost

Engine support varies; may scan more than a graph walk

Fails when

The filter is broad and the index is graph-based

  1. The filter is broad and the index is graph-based
  2. Filters are selective or correlated with distance
  3. Tenants are numerous and tiny
  4. The filter is a permission boundary

Figure. Each choice with what it buys, what it costs, and the case where it loses.

What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.

Text version

Four trade-offs: Pre-filtering is good for Correct result counts under selective filters, costs Engine support varies; may scan more than a graph walk, and fails when The filter is broad and the index is graph-based. Post-filtering with over-fetch is good for Works on any engine, costs Wasted candidates and unpredictable counts, and fails when Filters are selective or correlated with distance. Partition per tenant is good for Small search spaces and clean isolation, costs Many partitions and cross-tenant queries become awkward, and fails when Tenants are numerous and tiny. Filter as a soft boost is good for Never returns empty, costs Leaks ineligible documents, and fails when The filter is a permission boundary

Failure modes

  • Fewer results than requested. A selective filter leaves three candidates out of a requested ten, and the prompt is under-filled. Debug: log candidate counts before and after filtering for the narrowest filter in use.
  • Permission filter applied after ranking. An ineligible document is ranked, then dropped, and its presence still influenced the ordering or the cached answer. Debug: assert that the filter predicate is part of the query, then verify no ineligible id appears in the raw candidate list.
  • Filter value drift. Documents are indexed with an old tenant identifier and become invisible after a rename. Debug: count documents per filter value and compare with the source system.

In production

Monitor the distribution of eligible-result counts per filter and alert when it approaches zero. Keep filters inside the query plan, and rehearse the narrowest tenant on every index change.

Common misconceptions

People think: Filtering after the search is equivalent to filtering before it.

Actually: It changes both the result count and which documents are considered, so recall under selective filters differs.

People think: Metadata is just for display.

Actually: Filter fields are part of the index contract and determine which documents a query can reach.

Interview angle

Ask first whether the engine pre-filters, then design partitions and over-fetch around the answer. Treat permission filters as a correctness boundary, not a ranking preference. The weak version adds a metadata filter and never checks whether k results came back.

Sources