Why it matters
A coding session accumulates file reads, test output and failed attempts until the window is full. Truncating the oldest turns removes the task statement first, which is the one thing that must survive. Compaction exists to choose what to keep under a fixed budget, but every summary is a model-written lossy transformation, so the durable facts have to live somewhere the summary cannot corrupt.
Mental model
It is repacking a suitcase mid-trip and keeping what you need next. The metaphor breaks because you cannot tell which receipt is the one that proves the expense, and the summary is written by the same model that later relies on it.
How it works
- 61,400
- 9,200
- 700
- 1,600
- 900
- 1,100
- 300
- 4,600
- 52,200
- 38,000
- 12,000
- 2,200
Step 1 of 8
The window is nearly full
61,400 of 64,000 tokens are history, so the next read will not fit. The default response is to drop the oldest turns, which is the branch considered next.
root · current
Session · before compaction
Almost every token is spent on the session's own history. The next file read does not fit, so something has to leave before the run can continue.
Root of the hierarchy · 61,400 counted
Figure. A full session split into the 9.2k tokens that survive and the 52.2k that do not, with the task and its acceptance check inside the kept branch. The branch crossed at step 2 is sliding-window truncation, which drops the oldest turns first and therefore loses the objective before it loses the noise.
What to notice. The window and every token count are an authored example, not a measurement: 61.4k before, 9.2k kept and 52.2k dropped, and each branch sums to its parent. The page gives no totals, so the figure must be read for what is kept and why, not for its arithmetic. Omitted: the compaction prompt itself and the model's summary, which is the lossy step the scratch file exists to bound.
Text version
A compaction drawn as a hierarchy with token counts. The root is the session before compaction, 61,400 of a 64,000-token window. Two counted branches hang from it. Kept is 9,200 tokens: the task statement and acceptance check at 700, re-injected verbatim; decisions and reasons at 1,600; changed file paths and ranges at 900; the last failing output at 1,100, kept verbatim; open questions at 300; and recent turns at 4,600. Dropped is 52,200 tokens: successful tool output at 38,000, exploratory reads at 12,000 and resolved discussion at 2,200. Sliding-window truncation, which drops the oldest turns first, is drawn crossed and ruled out. A boundary gate says to compact after a passing test, not mid-edit, and a scratch file holds the durable state outside the summary. A second-compaction branch that summarises the previous summary is also crossed, because details decay and the loss cannot be seen from inside.
- Write durable state to a scratch file: task, plan, files changed, commands run and the current failure.
- Summarise completed steps into decisions and outcomes rather than transcripts of tool calls.
- Keep the most recent failing output verbatim, since the exact error text drives the next action.
- Drop successful tool output first, then exploratory reads, then resolved discussion.
- Re-inject the task statement and acceptance check immediately after compaction.
- Verify by asking the agent to restate the next step and comparing it with the scratch file.
Trade-offs
| Choice | Good for | Cost | Fails when |
|---|---|---|---|
| Summarise older turns | Keeps one continuous session | Lossy, can drop the key detail | Task depends on exact earlier output |
| Restart with a handoff file | Clean context, auditable | Handoff writing effort | State is hard to externalise |
| Sliding window truncation | Trivial to implement | Loses the task statement first | Session is long |
| Retrieve from a transcript store | No loss | Retrieval errors, extra latency | Search cannot find the right turn |
Good for
Keeps one continuous session
Cost
Lossy, can drop the key detail
Fails when
Task depends on exact earlier output
- Task depends on exact earlier output
- State is hard to externalise
- Session is long
- Search cannot find the right turn
Figure. Each choice with what it buys, what it costs, and the case where it loses.
What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.
Text version
Four trade-offs: Summarise older turns is good for Keeps one continuous session, costs Lossy, can drop the key detail, and fails when Task depends on exact earlier output. Restart with a handoff file is good for Clean context, auditable, costs Handoff writing effort, and fails when State is hard to externalise. Sliding window truncation is good for Trivial to implement, costs Loses the task statement first, and fails when Session is long. Retrieve from a transcript store is good for No loss, costs Retrieval errors, extra latency, and fails when Search cannot find the right turn
Failure modes
- Lost constraint. A user constraint from the first turn disappears and the agent violates it again. Debug: check the compacted context for the acceptance criteria and constraints block.
- Stale summary. The summary says tests pass though a later edit broke them. Debug: compare the summary's claims with the last command log.
- Summary drift. Each compaction paraphrases the previous summary and details decay. Debug: keep the original task text outside the summary chain.
In production
Log compaction events, token counts before and after, and task success by session length. Cap compactions per task and escalate to a written handoff when the cap is hit.
Common misconceptions
People think: Compaction is just truncation.
Actually: Truncation drops the oldest text, while compaction chooses what to keep, which is a modelling decision.
People think: A bigger context window removes the need.
Actually: Long contexts still degrade attention and raise cost per turn, so selection stays necessary.
Interview angle
Separate durable state from conversational state and name what must be kept verbatim. A strong answer verifies the summary against a scratch file and explains why the task statement is re-injected. Describing a conversation summary, with no mention of drift or loss, misses the failure that makes compaction dangerous.
Sources
- Anthropic, Effective context engineering for AI agents — compaction, structured notes and context budgets.
- Liu et al., Lost in the Middle — attention over long contexts and position effects.
- Packer et al., MemGPT — managing memory beyond a fixed context window.