Why it matters
A patch is wrong for one of two reasons: the model did not know something, or it knew something false. Both are context failures. A 200,000-token window does not solve this, because a mid-sized monorepo exceeds it by orders of magnitude, and because filling the window with loosely relevant code reduces precision even when it fits.
Good context engineering for code is mostly subtraction: a small, high-signal view of the repository, plus tools that let the agent ask for more. That is cheaper per turn, faster, and easier to debug than a giant preloaded prompt.
Mental model
A city map on the wall of a hire car. It shows the shape of the city, the main roads and the district names. It does not show every building. When you need a specific street, you zoom in on that district, and when you need a building, you look at the entrance.
The analogy breaks on fidelity. A map that is out of date sends you to a demolished street with full confidence. Code context has the same property: a repo map or embedding index built before a refactor will confidently point at files that no longer exist and miss the ones that do.
How it works
Step 1 of 7
Fixed layer
Instructions, project conventions and the task total 1,200 tokens and are marked non-evictable. A budget that can evict the objective turns the agent into a confident optimiser of the wrong thing.
included · current
fixed layer
System instructions, the repository's AGENTS.md and the task definition. Small, always present and marked non-evictable: budget pressure removes retrieval first, never the objective.
Under Edit · billing/refund.ts:112
Figure. A 60,000-token window with 9,600 committed: 1,200 fixed, 2,800 structural and 5,600 of specific code. The excluded branch is the other half of the decision — about 22,000 tokens of candidate material are deliberately not sent.
What to notice. Context is subtraction, not packing. Four included files cover the target, its type, one caller and one test; the rest of the window stays free for reads the agent chooses. Position is part of the allocation: the test sits near the end, where long-context use is strongest. Token counts are illustrative and rounded; the layer order and the non-evictable fixed layer are the page's mechanism.
Text version
A context allocation drawn as a hierarchy under one edit. The root is billing/refund.ts line 112 with a 60,000-token window and 9,600 tokens used. Four branches hang from it. The fixed layer is instructions, AGENTS.md and the task at 1,200 tokens, tagged pinned and not evictable. The structural map is a ranked 2,800-token slice. Included code is refund.ts at 2,100 tokens, types.ts at 400, plan.ts at 1,200 and refund.test.ts at 1,900. Excluded material is src/index.ts as a barrel re-export, legacy with seven files and no callers, fixtures at 18,000 tokens, docs/billing.md as prose, and git history, which was never requested; together they are about 22,000 tokens. A final step notes 50,400 tokens remain free for on-demand reads.
- Establish the fixed layer. System instructions, the project instruction file, and the task definition. These are small, always present, and should never be evicted; see AGENTS.md for the repository-side conventions.
- Add a structural summary. A repo map: file tree, top-level symbols, signatures, and edges between them, ranked by relevance to the task. This is the single highest-leverage context item for code, because it tells the model where to look.
- Add task-relevant retrieval. Two mechanisms are complementary: lexical or symbol search for exact identifiers, and semantic retrieval for intent. Code queries usually name identifiers exactly, which is why grep and LSP-based symbol lookup are the stronger default and embeddings are the supplement.
- Let the agent pull detail. Expose read tools with line ranges and search tools with context lines. Progressive disclosure keeps the window small and makes each addition attributable to a specific question.
- Order deliberately. Put the task and constraints at the start, the most relevant code near the end of the prompt, and bulk reference material in the middle or out of the prompt entirely.
- Recompute, do not cache blindly. Invalidate the map on file changes and record the commit hash it was built from.
- Measure. Log tokens per section and whether each retrieved item was read, cited in the patch, or ignored.
A concrete allocation for a mid-sized task might reserve a small fixed share for instructions and task, a moderate share for the repo map, and the remainder for specific files the agent requests. The exact split matters less than making it explicit and adjustable per task class.
def build_context(task, repo, budget_tokens=60_000):
ctx = Section("instructions", repo.instructions, priority=0, evictable=False)
ctx += Section("task", task.text, priority=0, evictable=False)
ctx += Section("repo_map", repo.map(top_k=120, query=task.text), priority=2)
ctx += Section("open_files", repo.recent_reads(window=5), priority=3)
ctx += Section("search_hits", repo.search(task.symbols(), context_lines=4), priority=4)
while ctx.tokens > budget_tokens:
victim = ctx.lowest_priority_evictable()
if victim is None:
ctx.truncate_largest(keep_head=40, keep_tail=200) # preserve signatures
break
ctx.remove(victim)
return ctx.render(order=["instructions", "task", "repo_map", "search_hits", "open_files"])Two decisions in that sketch are worth defending. evictable=False protects the task from budget pressure, which is the failure that turns an agent into a confident optimiser of the wrong objective. And truncate_largest keeps head and tail of a region, because signatures and closing logic usually matter more than the middle of a long function.
Trade-offs
| Context strategy | Good for | Cost | Fails when |
|---|---|---|---|
| Whole-file inclusion | Small files, unfamiliar code | Tokens, dilution, lost middle | The file is long or the repo is large |
| Repo map plus targeted reads | Most repository tasks | Map build time and staleness risk | The map does not model dynamic references |
| Semantic retrieval over chunks | Conceptual questions, docs | Misses exact identifiers, needs reindexing | The task names a symbol precisely |
| Symbol search via LSP | Refactors, definitions, references | Language server per language, cold start | The language has no server or the project does not resolve |
| Full conversation history | Short sessions | Quadratic token growth | The session passes twenty turns |
Good for
Small files, unfamiliar code
Cost
Tokens, dilution, lost middle
Fails when
The file is long or the repo is large
- The file is long or the repo is large
- The map does not model dynamic references
- The task names a symbol precisely
- The language has no server or the project does not resolve
- The session passes twenty turns
Figure. Each choice with what it buys, what it costs, and the case where it loses.
What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.
Text version
Four trade-offs: Whole-file inclusion is good for Small files, unfamiliar code, costs Tokens, dilution, lost middle, and fails when The file is long or the repo is large. Repo map plus targeted reads is good for Most repository tasks, costs Map build time and staleness risk, and fails when The map does not model dynamic references. Semantic retrieval over chunks is good for Conceptual questions, docs, costs Misses exact identifiers, needs reindexing, and fails when The task names a symbol precisely. Symbol search via LSP is good for Refactors, definitions, references, costs Language server per language, cold start, and fails when The language has no server or the project does not resolve. Full conversation history is good for Short sessions, costs Quadratic token growth, and fails when The session passes twenty turns
Failure modes
- Stale map. Files were renamed and the map still lists the old ones, so the agent reads a path that fails and concludes the code is missing. Debug: pin the map to a commit hash and compare it with HEAD before the session starts.
- Retrieval by embedding only. The task says
parseInvoiceTotaland the top chunks are prose about invoices. Debug: log whether exact-identifier search ran; add a lexical pass and merge results before the model sees them. - Middle burial. The one relevant hunk sits between two long files and is ignored. Debug: for failing runs, check whether the decisive hunk was in the prompt and where; if it was mid-prompt, reorder rather than retrieve more.
- Context bloat from tool output. A single
catof a generated file consumes the budget. Debug: cap bytes per read and require line ranges for files above a threshold.
In production
Context assembly is the main lever on cost per task, because input tokens dominate. A session that re-sends 60k tokens twenty times is a large bill for one change. Compaction, caching of stable prefixes and subagent isolation all attack the same number; see compaction.
Watch map build time separately: on large repositories it can exceed the first model call. Cache it per commit, invalidate on write, and fall back to a file-tree-only map rather than delaying the session. Also track context utilisation: a model given 8k of a 200k window may under-use it, and a model given 190k may lose the middle. Both are visible only if you log what was sent.
Common misconceptions
People think: A bigger context window removes the need for retrieval.
Actually: It raises the ceiling on what fits, not on what is used well. Long contexts still suffer position effects and cost scales with every token sent.
People think: Embeddings are the right retrieval method for code.
Actually: Code queries are usually exact identifiers and structural relationships. Lexical and symbol search are stronger defaults; embeddings help with intent-level questions and documentation.
Interview angle
Expect "how would you give a coding agent access to a large repository?" Answer with the hierarchy: fixed instructions, structural map, on-demand reads, exact-identifier search first. Then discuss budget and position effects, and how you would detect that context, not reasoning, caused a failure. See agentic coding interview questions, repo maps and context engineering.
Sources
- Aider, Repository map — a concrete implementation of a ranked, token-budgeted structural summary using tree-sitter and a graph ranking.
- Liu et al., Lost in the Middle: How Language Models Use Long Contexts — evidence that retrieval quality depends on position, not just presence.
- Anthropic, Effective context engineering for AI agents — progressive disclosure and compaction as context strategies.
- Microsoft, Language Server Protocol specification — the interface behind definition and reference lookups.