Why it matters
A repository is larger than any context window, and the first decision an agent makes is which files to open. Lexical search finds exact names but misses concepts that were renamed. Embedding search finds related code but is weak on identifiers. Neither tells the agent when to stop. Discovery exists to turn an unbounded repository into a short, justified reading list before any edit is attempted.
Mental model
It is a library with no catalogue: you start at the returns cart, follow citations from one book to the next and stop when the shelf looks right. The metaphor breaks because the agent cannot see that a volume is missing, only that its query returned nothing, which looks the same as a book that was never written.
How it works
Step 1 of 8
Identifiers extracted
The route, the status and the plan name are pulled from the report before any search runs. The frame is unknown, so the route string is the first query.
root · current
Issue · 500 on POST /api/refunds
The report names a route, a status and a plan name. No function, no frame. The route string is the strongest identifier available, so discovery starts there.
Root of the hierarchy
Figure. One issue, three search branches and a reading list of four files. The exact identifier wins; the embedding branch and the prose grep are recorded as rejected, which is what makes the short list defensible.
What to notice. Discovery has a budget, so the trace shows rejected branches as well as the ones that worked. Each rejection is a search that cost context and a reason it stopped. Files, hit counts and the 6.1k token total are illustrative for a small billing feature; the order — identifiers, manifest, import hop, nearest test — is the page's procedure.
Text version
A search trace rendered as a hierarchy. The root is an issue: a 500 on POST /api/refunds for annual plans, with no stack trace. Four branches hang from it. The first, a grep for period, is marked rejected with zero hits in src. The second, a grep for refundWindow, is marked selected with three files and leads to src/billing/refund.ts, then one import hop to plan.ts and types.ts, and to the nearest test, tests/refund.test.ts, read first. The third branch, an embedding search for annual plan refund, is rejected because it returns five prose chunks with no identifier. A manifest branch shows package.json and vitest with a single src/billing feature directory. The final step records four files read and two search branches rejected: the period grep and the embedding search.
- Extract identifiers from the task text first, including error strings, function names, config keys and route paths.
- Run lexical search for each identifier and keep the file list rather than only the matching snippets.
- Read the package manifest and build config to learn the language, test runner and module layout.
- Follow imports outward from the strongest hit one hop at a time, stopping when the call graph leaves the feature.
- Find the test nearest to the behaviour and read it before editing anything.
- Write down the files considered and rejected so a failed run shows where selection went wrong.
Trade-offs
| Choice | Good for | Cost | Fails when |
|---|---|---|---|
| Lexical grep | Exact identifiers, fast, explainable | Misses synonyms and renames | Code uses generated names or the task is conceptual |
| Embedding search | Finds conceptually related code | Index staleness, weak on identifiers | Repository changes every hour |
| Symbol index or language server | Precise definitions and references | Needs a working server per language | Monorepo mixes supported and unsupported languages |
| Repo map summary | Cheap orientation on large repos | Summaries omit the line that matters | Task hinges on a specific expression |
Good for
Exact identifiers, fast, explainable
Cost
Misses synonyms and renames
Fails when
Code uses generated names or the task is conceptual
- Code uses generated names or the task is conceptual
- Repository changes every hour
- Monorepo mixes supported and unsupported languages
- Task hinges on a specific expression
Figure. Each choice with what it buys, what it costs, and the case where it loses.
What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.
Text version
Four trade-offs: Lexical grep is good for Exact identifiers, fast, explainable, costs Misses synonyms and renames, and fails when Code uses generated names or the task is conceptual. Embedding search is good for Finds conceptually related code, costs Index staleness, weak on identifiers, and fails when Repository changes every hour. Symbol index or language server is good for Precise definitions and references, costs Needs a working server per language, and fails when Monorepo mixes supported and unsupported languages. Repo map summary is good for Cheap orientation on large repos, costs Summaries omit the line that matters, and fails when Task hinges on a specific expression
Failure modes
- Wrong neighbourhood. The agent edits a plausible but unrelated module and tests still pass because nothing covers it. Debug: print the discovery trace and check whether the reported error string appears in any edited file.
- Empty search conclusion. Search returns nothing and the agent creates a duplicate helper. Debug: re-run with the identifier split into words and check whether ignore rules excluded the path.
- Context spent on reading. Large files consume the window and leave no room to patch. Debug: log bytes read per file and cap file size before reading.
In production
Log the discovery trace per task and review which files were rejected. Alert when a run reads more than a set number of files without producing a patch, which usually means the search scope is wrong.
Common misconceptions
People think: Semantic search replaces grep.
Actually: Exact identifiers are the strongest signal in code; embeddings help mainly when the task is described in prose.
People think: The agent should read the whole repository.
Actually: Reading everything spends the context needed to write and verify, so selection is the skill being tested.
Interview angle
Walk through how you would find the file for a reported error, starting from the identifier and ending at a test. A strong answer names the signals, the stopping rule and the trace that shows which files were rejected. Naming semantic search as the whole strategy, with no explanation of how selection is verified, misses why discovery is where most runs fail.
Sources
- Aider, Repository map — a production repo map built from a symbol graph.
- Liu et al., RepoBench — repository-level code completion and cross-file context.
- ripgrep — the lexical search primitive most agents use.