staffReportedcurated
Each user can see under 1% of a 50M-document corpus. What happens if you apply the permission filter after ANN search, and what do you do instead?
30-second answer
- Post-filtering collapses here: retrieving 50 candidates where a user sees 1% of the corpus yields about 0.5 visible chunks, so returning five needs roughly 500 over-fetched candidates.
- Every one of those candidates came from an unrestricted graph walk, so restricted content has already been retrieved and scored, one caching bug away from a leak.
- Pre-filter inside the index with tenant-aware partitioning or per-tenant subgraphs, and fall back to brute-force scan when the allowed set is tiny.
- A filter excluding 99% of vectors breaks HNSW connectivity, because most neighbours the walk would traverse are excluded, so recall drops and traversal explodes.
- Benchmark filtered recall against exact search at the production k, since unfiltered recall says nothing about the narrow tenant that matters.
Likely follow-ups
Open a question to rehearse the answer.
A user's group membership changes mid-session, how quickly must the filter reflect it?
- Treat revocation as a freshness problem with a stated target, propagated to the index and to every cache on the same clock.
- Hold the permission set in the verified identity or a short-TTL lookup, so a change takes effect without a re-index.
- Re-authorise at each request and at tool-call time, because context assembled under the old grant outlives the grant itself.
When is post-filtering actually acceptable, and what has to be true about the visibility ratio?
- When the filter admits most of the corpus, so over-fetching by a small factor restores k and recall barely moves.
- When the field is a preference rather than a permission boundary, such as recency or region, since then a dropped result leaks nothing.
- It stops being acceptable once the eligible share is small or correlated with distance, because result counts and recall both become unpredictable.
How do you benchmark filtered recall when the allowed set is 1% of the corpus?
- Build exact-search ground truth restricted to the same eligible set, then measure overlap at the production k against the filtered index.
- Sample real tenants or users rather than a synthetic filter, because filter cardinality is what drives the result.
- Log pre-filter and post-filter candidate counts per query, and alert when the returned count falls short of k.
Red flags
- prompt as control
- unfiltered recall as evidence
Concepts this tests
Tags
Source: github.com
More in RAG & retrieval pipelines
- Explain parent-child (small-to-big) retrieval. What problem does it solve and what does it cost you?
- Explain HNSW. What do M, ef_construction and ef_search each control, and which can you change in production?
- How would you tune HNSW for 10 million vectors when the product needs p95 retrieval under 100 ms?