Why it matters
A long-running assistant that forgets the user's constraints every session is unusable, but stuffing every past turn into the prompt is expensive and dilutes attention. The engineering problem is deciding what to persist, in what shape, and how it is recalled. Without a write policy memory accumulates contradictions; without invalidation a corrected fact competes with the stale one.
Mental model
Memory is a notebook the agent keeps beside the conversation, not a recording of it. The metaphor breaks because a notebook has an author who chooses what to write; recording everything automatically turns the notebook into a transcript and recall into search over noise.
How it works
Keyed transitions
- Admit → Store: source ∧ owner ∧ ttl
Event · step 1 of 8
Run starts
Entry: a new session with an empty working context. The store holds six current records under user 8412; none of them is in the prompt yet.
In state
- Run
Next: Task ends
- Current records
- 6
- Superseded
- 0
- Injected
- 0
Figure. Eight states. Admission requires a source turn, an owner and an expiry; recall filters superseded records and caps how many enter the prompt. Stale is the failure the page warns about: a record recalled outside the scope that justified it.
What to notice. The working context and the durable store are separate: nothing survives a run unless the write gate admits it. A superseding write adds a record and marks the old one non-current instead of appending a contradiction. Record counts, the five-record cap, the 180-day expiry, the 0.6 relevance cut-off and the 180-token injection are illustrative. A poisoned record that passes the source check is not a separate state; the failure drawn here is the scope leak, and encryption and per-field redaction are omitted.
Text version
A state machine with eight states. Run goes to Extract on task end; Extract goes to Admit on a candidate fact. Admit goes to Store when a source, owner and expiry are present. Store goes to Recall on load, and to Deleted on expiry or user deletion. Recall goes to Context when five or fewer records are relevant and none is superseded, and to Stale when the scope check is missing. Context loops back to Run on the next run. The trace walks Run, Extract, Admit, Store, Recall, Context, Deleted, then Stale. Counters show six current records, one superseded and one injected record.
- Separate working context, rebuilt each run, from durable memory, loaded on demand.
- Define records with a type, a source, a timestamp, a confidence and an owner.
- Write deliberately: extract facts after a task, deduplicate against existing records, and store the source turn id.
- Retrieve by task relevance rather than recency alone, and cap how many records enter the prompt.
- Supersede on contradiction instead of appending, and honour user deletion requests.
- Log which memory records were injected into each prompt so a wrong answer can be traced.
Trade-offs
| Choice | Good for | Cost | Fails when |
|---|---|---|---|
| Full transcript replay | No extraction loss | Cost grows with history, attention dilutes | Sessions run long |
| Extracted fact store | Compact and queryable | Extraction errors and lost nuance | The task depends on exact wording |
| Per-user long-term memory | Personalisation across sessions | Privacy surface and deletion obligations | Several users share a device or account |
| Session-only memory | Simple and private | No continuity | Continuity is the product's value |
Good for
No extraction loss
Cost
Cost grows with history, attention dilutes
Fails when
Sessions run long
- Sessions run long
- The task depends on exact wording
- Several users share a device or account
- Continuity is the product's value
Figure. Each choice with what it buys, what it costs, and the case where it loses.
What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.
Text version
Four trade-offs: Full transcript replay is good for No extraction loss, costs Cost grows with history, attention dilutes, and fails when Sessions run long. Extracted fact store is good for Compact and queryable, costs Extraction errors and lost nuance, and fails when The task depends on exact wording. Per-user long-term memory is good for Personalisation across sessions, costs Privacy surface and deletion obligations, and fails when Several users share a device or account. Session-only memory is good for Simple and private, costs No continuity, and fails when Continuity is the product's value
Failure modes
- Memory poisoning. A wrong or injected fact is stored and recalled confidently later. Debug: sample stored records and check each has a verifiable source turn.
- Stale contradiction. A superseded preference is retrieved alongside the new one and the agent alternates. Debug: query records for the same key and assert one is marked current.
- Recall flooding. Too many records enter the prompt and the relevant one is ignored. Debug: log injected record ids and compare answer quality at a smaller cap.
In production
Monitor records written per run, retrieval hit rate and prompt tokens spent on memory. Cap records per prompt and per user, and keep a deletion path that you actually test.
Common misconceptions
People think: A larger context window removes the need for memory.
Actually: A window is per-run and priced per token; persistence, invalidation and privacy remain your problem.
People think: Memory should store everything so nothing is lost.
Actually: Unfiltered storage makes recall worse and creates data you must later delete.
Interview angle
Strong answers separate the context window from the store, describe the write policy and the invalidation rule, and treat recall as retrieval with its own evaluation. Weak answers say the agent has a vector database of past conversations and never address what gets written or removed.
Sources
- Packer et al., MemGPT: Towards LLMs as Operating Systems — managing a bounded context against an external memory store.
- Park et al., Generative Agents: Interactive Simulacra of Human Behavior — a memory stream with retrieval, reflection and recency weighting.
- Sumers et al., Cognitive Architectures for Language Agents — working, episodic, semantic and procedural memory as distinct stores.