Why it matters
Long runs fail: a deploy restarts the worker, a tool times out, a reviewer wants a change. Without a snapshot, recovery means replaying every step, which repeats side effects and costs tokens. A checkpoint makes the run's position a durable fact, so continuation is cheap and the state at any point can be examined after an incident.
Mental model
A checkpoint is a save file with a version number, not a screenshot of the screen. The metaphor breaks because a save file assumes the game is unchanged, while agent checkpoints are read by newer code, so the reader must migrate old state rather than trust it.
How it works
Keyed transitions
- Checkpoint · step 3 → Continue · step 4: valid
Event · step 1 of 7
Run at step 3
A tool result is in hand but not yet snapshotted. The action journal already has the external artifact id.
In state
- Run · step 3
Next: Checkpoint written
- Committed replayed
- 0
- In-flight re-issued
- 0
Figure. A checkpoint written at step 3 survives a worker crash. Restore reads it, replay skips the three committed steps, and the one call that was in flight is re-issued under its original idempotency key. The effect runs once because the tool deduplicates the key.
What to notice. The snapshot carries position and references, not effects. Committed steps are never replayed; the single in-flight call is re-issued and deduplicated by key. Without that key the resume would duplicate the call, which is the page's torn-checkpoint failure.
Text version
A state machine with seven states. Run at step 3 goes to Checkpoint at step 3, drawn with a teal marker. Checkpoint goes to Continue at step 4 on consistent. Continue goes to Crash on crash. Crash goes to Restore on load. Restore goes to Replay on validate. Replay returns to Continue on resume. Continue also goes to Done on complete. The trace walks Run, Checkpoint, Continue, Crash, Restore, Replay and Continue again. Counters show committed effects replayed staying at zero and in-flight effects re-issued reaching one, because the step-4 call was already dispatched when the worker died.
- Choose boundaries: after a tool result is recorded, before a side effect, and at each state transition.
- Write the checkpoint atomically with the action journal so state and effects cannot disagree.
- Include a schema version, the run id, the step index and the ids of external artifacts.
- Keep the payload small by referencing artifacts instead of embedding their contents.
- On resume, validate the schema and re-check external preconditions before continuing.
- Expire old checkpoints by retention policy, keeping the ones attached to open incidents.
Trade-offs
| Choice | Good for | Cost | Fails when |
|---|---|---|---|
| Checkpoint every step | Fine-grained recovery and forking | Write volume and storage cost | Runs have thousands of steps |
| Checkpoint at transitions | Cheap and semantically meaningful | Loses intra-step progress | A single step is long and expensive |
| Full state snapshot | Simple to restore | Large payloads and slow writes | State includes big artifacts |
| Reference-based state | Small and fast | Dangling references when artifacts expire | Artifact retention is shorter than run retention |
Good for
Fine-grained recovery and forking
Cost
Write volume and storage cost
Fails when
Runs have thousands of steps
- Runs have thousands of steps
- A single step is long and expensive
- State includes big artifacts
- Artifact retention is shorter than run retention
Figure. Each choice with what it buys, what it costs, and the case where it loses.
What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.
Text version
Four trade-offs: Checkpoint every step is good for Fine-grained recovery and forking, costs Write volume and storage cost, and fails when Runs have thousands of steps. Checkpoint at transitions is good for Cheap and semantically meaningful, costs Loses intra-step progress, and fails when A single step is long and expensive. Full state snapshot is good for Simple to restore, costs Large payloads and slow writes, and fails when State includes big artifacts. Reference-based state is good for Small and fast, costs Dangling references when artifacts expire, and fails when Artifact retention is shorter than run retention
Failure modes
- Torn checkpoint. The snapshot was written while a tool call was in flight and resume duplicates it. Debug: compare the checkpoint's step index with the journal's last committed action.
- Schema drift. New code cannot read an old checkpoint and the run is lost. Debug: keep a migration test that loads a stored fixture from the previous schema version.
- Dangling reference. The checkpoint points at an artifact that was cleaned up. Debug: verify referenced objects exist as part of resume and fail loudly when they do not.
In production
Monitor checkpoint write latency, size and resume success rate. Cap steps between checkpoints and test restore from a real production checkpoint after every schema change.
Common misconceptions
People think: Checkpoints are the same as database transactions.
Actually: A checkpoint records the run's position; it does not make external effects atomic with it.
People think: Checkpointing every token is safest.
Actually: Frequent writes cost latency and storage; boundaries chosen for consistency matter more than frequency.
Interview angle
Strong answers put checkpoints at consistent boundaries, discuss schema versioning and distinguish a checkpoint from a rollback. Weak answers say state is saved to Redis and never mention what happens when the saved shape changes.
Sources
- LangGraph documentation — persisted graph state and checkpointing for resumable runs.
- Temporal documentation — event history as the durable record a workflow replays against.