Skip to content
AI Engineer Guide.

concept · coding · intermediate · concept 24 of 30

Compaction

0 of 30 complete

Why it matters

A coding session accumulates file reads, test output and failed attempts until the window is full. Truncating the oldest turns removes the task statement first, which is the one thing that must survive. Compaction exists to choose what to keep under a fixed budget, but every summary is a model-written lossy transformation, so the durable facts have to live somewhere the summary cannot corrupt.

Mental model

It is repacking a suitcase mid-trip and keeping what you need next. The metaphor breaks because you cannot tell which receipt is the one that proves the expense, and the summary is written by the same model that later relies on it.

How it works

What compaction keeps, what it dropsInteractive
16 nodes · 3 branches
  • 61,400
    • 9,200
      • 700
      • 1,600
      • 900
      • 1,100
      • 300
      • 4,600
    • 52,200
      • 38,000
      • 12,000
      • 2,200
includedexcludedvalidation gatenoterootfileruled out or rejectedbar = tokens · parent counts include their groups · scaled to the full session (61.4k)

Step 1 of 8

The window is nearly full

61,400 of 64,000 tokens are history, so the next read will not fit. The default response is to drop the oldest turns, which is the branch considered next.

root · current

Session · before compaction

Almost every token is spent on the session's own history. The next file read does not fit, so something has to leave before the run can continue.

Root of the hierarchy · 61,400 counted

Step 1/8 · The window is nearly full

Figure. A full session split into the 9.2k tokens that survive and the 52.2k that do not, with the task and its acceptance check inside the kept branch. The branch crossed at step 2 is sliding-window truncation, which drops the oldest turns first and therefore loses the objective before it loses the noise.

What to notice. The window and every token count are an authored example, not a measurement: 61.4k before, 9.2k kept and 52.2k dropped, and each branch sums to its parent. The page gives no totals, so the figure must be read for what is kept and why, not for its arithmetic. Omitted: the compaction prompt itself and the model's summary, which is the lossy step the scratch file exists to bound.

Text version

A compaction drawn as a hierarchy with token counts. The root is the session before compaction, 61,400 of a 64,000-token window. Two counted branches hang from it. Kept is 9,200 tokens: the task statement and acceptance check at 700, re-injected verbatim; decisions and reasons at 1,600; changed file paths and ranges at 900; the last failing output at 1,100, kept verbatim; open questions at 300; and recent turns at 4,600. Dropped is 52,200 tokens: successful tool output at 38,000, exploratory reads at 12,000 and resolved discussion at 2,200. Sliding-window truncation, which drops the oldest turns first, is drawn crossed and ruled out. A boundary gate says to compact after a passing test, not mid-edit, and a scratch file holds the durable state outside the summary. A second-compaction branch that summarises the previous summary is also crossed, because details decay and the loss cannot be seen from inside.

  1. Write durable state to a scratch file: task, plan, files changed, commands run and the current failure.
  2. Summarise completed steps into decisions and outcomes rather than transcripts of tool calls.
  3. Keep the most recent failing output verbatim, since the exact error text drives the next action.
  4. Drop successful tool output first, then exploratory reads, then resolved discussion.
  5. Re-inject the task statement and acceptance check immediately after compaction.
  6. Verify by asking the agent to restate the next step and comparing it with the scratch file.

Trade-offs

ChoiceGood forCostFails when
Summarise older turnsKeeps one continuous sessionLossy, can drop the key detailTask depends on exact earlier output
Restart with a handoff fileClean context, auditableHandoff writing effortState is hard to externalise
Sliding window truncationTrivial to implementLoses the task statement firstSession is long
Retrieve from a transcript storeNo lossRetrieval errors, extra latencySearch cannot find the right turn
Trade-off mapInteractive

Good for

Keeps one continuous session

Cost

Lossy, can drop the key detail

Fails when

Task depends on exact earlier output

  1. Task depends on exact earlier output
  2. State is hard to externalise
  3. Session is long
  4. Search cannot find the right turn

Figure. Each choice with what it buys, what it costs, and the case where it loses.

What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.

Text version

Four trade-offs: Summarise older turns is good for Keeps one continuous session, costs Lossy, can drop the key detail, and fails when Task depends on exact earlier output. Restart with a handoff file is good for Clean context, auditable, costs Handoff writing effort, and fails when State is hard to externalise. Sliding window truncation is good for Trivial to implement, costs Loses the task statement first, and fails when Session is long. Retrieve from a transcript store is good for No loss, costs Retrieval errors, extra latency, and fails when Search cannot find the right turn

Failure modes

  • Lost constraint. A user constraint from the first turn disappears and the agent violates it again. Debug: check the compacted context for the acceptance criteria and constraints block.
  • Stale summary. The summary says tests pass though a later edit broke them. Debug: compare the summary's claims with the last command log.
  • Summary drift. Each compaction paraphrases the previous summary and details decay. Debug: keep the original task text outside the summary chain.

In production

Log compaction events, token counts before and after, and task success by session length. Cap compactions per task and escalate to a written handoff when the cap is hit.

Common misconceptions

People think: Compaction is just truncation.

Actually: Truncation drops the oldest text, while compaction chooses what to keep, which is a modelling decision.

People think: A bigger context window removes the need.

Actually: Long contexts still degrade attention and raise cost per turn, so selection stays necessary.

Interview angle

Separate durable state from conversational state and name what must be kept verbatim. A strong answer verifies the summary against a scratch file and explains why the task statement is re-injected. Describing a conversation summary, with no mention of drift or loss, misses the failure that makes compaction dangerous.

Sources