Skip to content
AI Engineer Guide.

concept · coding · intermediate · concept 4 of 30

Repo discovery

0 of 30 complete

Why it matters

A repository is larger than any context window, and the first decision an agent makes is which files to open. Lexical search finds exact names but misses concepts that were renamed. Embedding search finds related code but is weak on identifiers. Neither tells the agent when to stop. Discovery exists to turn an unbounded repository into a short, justified reading list before any edit is attempted.

Mental model

It is a library with no catalogue: you start at the returns cart, follow citations from one book to the next and stop when the shelf looks right. The metaphor breaks because the agent cannot see that a volume is missing, only that its query returned nothing, which looks the same as a book that was never written.

How it works

Three branches tried, one reading listInteractive
9 nodes · 3 branches
fileselectedroottest suiteruled out or rejected

Step 1 of 8

Identifiers extracted

The route, the status and the plan name are pulled from the report before any search runs. The frame is unknown, so the route string is the first query.

root · current

Issue · 500 on POST /api/refunds

The report names a route, a status and a plan name. No function, no frame. The route string is the strongest identifier available, so discovery starts there.

Root of the hierarchy

Step 1/8 · Identifiers extracted

Figure. One issue, three search branches and a reading list of four files. The exact identifier wins; the embedding branch and the prose grep are recorded as rejected, which is what makes the short list defensible.

What to notice. Discovery has a budget, so the trace shows rejected branches as well as the ones that worked. Each rejection is a search that cost context and a reason it stopped. Files, hit counts and the 6.1k token total are illustrative for a small billing feature; the order — identifiers, manifest, import hop, nearest test — is the page's procedure.

Text version

A search trace rendered as a hierarchy. The root is an issue: a 500 on POST /api/refunds for annual plans, with no stack trace. Four branches hang from it. The first, a grep for period, is marked rejected with zero hits in src. The second, a grep for refundWindow, is marked selected with three files and leads to src/billing/refund.ts, then one import hop to plan.ts and types.ts, and to the nearest test, tests/refund.test.ts, read first. The third branch, an embedding search for annual plan refund, is rejected because it returns five prose chunks with no identifier. A manifest branch shows package.json and vitest with a single src/billing feature directory. The final step records four files read and two search branches rejected: the period grep and the embedding search.

  1. Extract identifiers from the task text first, including error strings, function names, config keys and route paths.
  2. Run lexical search for each identifier and keep the file list rather than only the matching snippets.
  3. Read the package manifest and build config to learn the language, test runner and module layout.
  4. Follow imports outward from the strongest hit one hop at a time, stopping when the call graph leaves the feature.
  5. Find the test nearest to the behaviour and read it before editing anything.
  6. Write down the files considered and rejected so a failed run shows where selection went wrong.

Trade-offs

ChoiceGood forCostFails when
Lexical grepExact identifiers, fast, explainableMisses synonyms and renamesCode uses generated names or the task is conceptual
Embedding searchFinds conceptually related codeIndex staleness, weak on identifiersRepository changes every hour
Symbol index or language serverPrecise definitions and referencesNeeds a working server per languageMonorepo mixes supported and unsupported languages
Repo map summaryCheap orientation on large reposSummaries omit the line that mattersTask hinges on a specific expression
Trade-off mapInteractive

Good for

Exact identifiers, fast, explainable

Cost

Misses synonyms and renames

Fails when

Code uses generated names or the task is conceptual

  1. Code uses generated names or the task is conceptual
  2. Repository changes every hour
  3. Monorepo mixes supported and unsupported languages
  4. Task hinges on a specific expression

Figure. Each choice with what it buys, what it costs, and the case where it loses.

What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.

Text version

Four trade-offs: Lexical grep is good for Exact identifiers, fast, explainable, costs Misses synonyms and renames, and fails when Code uses generated names or the task is conceptual. Embedding search is good for Finds conceptually related code, costs Index staleness, weak on identifiers, and fails when Repository changes every hour. Symbol index or language server is good for Precise definitions and references, costs Needs a working server per language, and fails when Monorepo mixes supported and unsupported languages. Repo map summary is good for Cheap orientation on large repos, costs Summaries omit the line that matters, and fails when Task hinges on a specific expression

Failure modes

  • Wrong neighbourhood. The agent edits a plausible but unrelated module and tests still pass because nothing covers it. Debug: print the discovery trace and check whether the reported error string appears in any edited file.
  • Empty search conclusion. Search returns nothing and the agent creates a duplicate helper. Debug: re-run with the identifier split into words and check whether ignore rules excluded the path.
  • Context spent on reading. Large files consume the window and leave no room to patch. Debug: log bytes read per file and cap file size before reading.

In production

Log the discovery trace per task and review which files were rejected. Alert when a run reads more than a set number of files without producing a patch, which usually means the search scope is wrong.

Common misconceptions

People think: Semantic search replaces grep.

Actually: Exact identifiers are the strongest signal in code; embeddings help mainly when the task is described in prose.

People think: The agent should read the whole repository.

Actually: Reading everything spends the context needed to write and verify, so selection is the skill being tested.

Interview angle

Walk through how you would find the file for a reported error, starting from the identifier and ending at a test. A strong answer names the signals, the stopping rule and the trace that shows which files were rejected. Naming semantic search as the whole strategy, with no explanation of how selection is verified, misses why discovery is where most runs fail.

Sources