Skip to content
AI Engineer Guide.

concept · coding · intermediate · concept 7 of 30

Code search (grep/symbol/AST/LSP)

0 of 30 complete

Why it matters

Agents ask several different questions of a codebase: where is this string, where is this symbol defined, what type does this expression have. No single tool answers all three. Grep is exact and noisy, symbol indexes are precise but need a build, and language servers are rich but heavy. Routing a question to the right tool, then ranking its output, decides whether the agent reads the right file.

Mental model

It is a toolbox holding a torch, a magnet and a blueprint. The metaphor breaks because all three can point at different objects for the same word, and none of them says which object the task actually needed, so the agent must read the hit before trusting it.

How it works

One question, one branch, then readInteractive
12 nodes · 2 branches
validation gatebranch triedfilerootnoteruled out or rejected

Step 1 of 7

Route by shape

The query is a quoted error string, so lexical search runs first; a question about where the name is defined would start at the symbol index, and a type question at the language server. The wrong route costs time and a reading list, not just tokens.

root · current

Query · "refundWindow is not a function"

The question quotes the error string. That shape decides the first tool; a question about where the name is defined would start at the symbol index, and a type question at the language server.

Root of the hierarchy

Also in focus

  • grep · "refundWindow"
  • symbol index · refundWindow
  • language server · period
Step 1/7 · Route by shape

Figure. One code question routed to the tool that matches its shape: a quoted string to grep, an identifier to the symbol index, a type to the language server. The branch that answers splits into the three files it returned; the embedding branch is tried for the concept and ruled out because it returns a similar passage, not the declaration or its callers.

What to notice. The query, the package and the three hits are an authored example; the page publishes no hit rates or timings, so none are drawn. The claim is routing, not ranking arithmetic: which tool answers which question, and what has to be read before the hit becomes evidence. Omitted: the language server's setup and the symbol index build, which the page covers as costs rather than as steps.

Text version

A code search trace drawn as a hierarchy. The root is the query 'refundWindow is not a function', a quoted error string routed by shape. A scope gate narrows the search to src/billing before any tool runs. The lexical branch, a grep for the quoted refundWindow, is tried and returns three files: refund.ts at line 41 as the definition, plan.ts at line 88 as the caller, and refund.test.ts as the nearest test. Under that branch, a rank gate keeps path proximity and the test before returning, and a read gate requires a full read before any edit. Two sibling branches answer different questions and are not chosen here: the symbol index answers where the name is defined, and the language server answers what type the expression has, both needing a build. An embedding search is tried for the concept and ruled out because it returns a similar passage, not the definition or its callers; a route log records tool, query, hits and whether the file was edited.

  1. Route the query by shape: a quoted string to grep, an identifier to symbol search, a type question to the language server.
  2. Scope the search to the package implied by the task before running it across the repository.
  3. Rank hits by path proximity to the entry point and by whether the file is a test.
  4. Apply ignore rules that match the build so generated and vendored paths do not flood the results.
  5. Return file, line and a short context window, then require a full read before editing.
  6. Record which tool answered each sub-question so a miss can be attributed later.

Trade-offs

ChoiceGood forCostFails when
ripgrepFast, exact, works on any fileNo semantics, noisy on common wordsIdentifier is short or appears in fixtures
Symbol indexPrecise definitions and referencesBuild or index step per commitCode is dynamic or the index is stale
Language serverTypes, overloads, diagnosticsHeavy startup, per-language setupRepository does not compile
AST pattern searchStructural matches across formattingQuery language to learnPattern spans macros or generated code
Trade-off mapInteractive

Good for

Fast, exact, works on any file

Cost

No semantics, noisy on common words

Fails when

Identifier is short or appears in fixtures

  1. Identifier is short or appears in fixtures
  2. Code is dynamic or the index is stale
  3. Repository does not compile
  4. Pattern spans macros or generated code

Figure. Each choice with what it buys, what it costs, and the case where it loses.

What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.

Text version

Four trade-offs: ripgrep is good for Fast, exact, works on any file, costs No semantics, noisy on common words, and fails when Identifier is short or appears in fixtures. Symbol index is good for Precise definitions and references, costs Build or index step per commit, and fails when Code is dynamic or the index is stale. Language server is good for Types, overloads, diagnostics, costs Heavy startup, per-language setup, and fails when Repository does not compile. AST pattern search is good for Structural matches across formatting, costs Query language to learn, and fails when Pattern spans macros or generated code

Failure modes

  • Grep noise. The agent reads twenty irrelevant matches and misses the real one. Debug: inspect the result list and add path or word-boundary filters before the next query.
  • Index behind the branch. Symbol search points at a definition deleted on this branch. Debug: compare the index commit with HEAD before trusting references.
  • Ignored-file blindness. The answer lives in a generated or vendored path excluded by ignore rules. Debug: run the same query with ignores disabled and diff the result sets.

In production

Log tool, query, hit count and whether the chosen file was edited. Alert on repeated zero-hit queries, which usually means the scope or the tool routing is wrong.

Common misconceptions

People think: Semantic code search removes the need for exact search.

Actually: Identifiers, error strings and config keys are best matched literally, and most agent queries start with one.

People think: More results means better recall.

Actually: Past the context budget, extra hits push out the file that actually mattered.

Interview angle

Map each question type to a tool before discussing any product. A strong answer covers literal search for identifiers, symbol lookup for definitions and a language server for types, then says what happens when the build is broken. A single-tool answer that ignores ranking and ignore rules will not survive a large repository.

Sources