Why it matters
Caching is the cheapest way to remove load and latency, and the reason it goes wrong is that the key and the invalidation rule are decided once and then the system changes around them. A new personalisation field, a permission check or a model version makes the stored copy wrong, and nothing fails loudly: users simply see yesterday's answer. Invalidation defines, per cached object, what makes it stale and who says so.
Mental model
It is a fridge with leftovers: the date label decides whether it is still good, and a smell is a late warning. The analogy breaks because a cache can be invalidated exactly when the source changes, while a fridge label is only ever an estimate.
How it works
- 1Write the key from every input that changes the response: path, query, tenant, locale, permission scope, model or prompt version.
- 2Choose a freshness policy per object: a lifetime for tolerable staleness, validators for revalidation, explicit purge for correctness-critical data.
- 3Set the lifetime from how fast the source changes, and record the choice next to the cache definition.
- 4Invalidate by version rather than by deleting keys, so a deploy changes the key space and old entries age out.
- 5Add request coalescing or a short lock so a popular expired key does not cause a stampede.
- 6Serve stale on origin failure only if the product accepts it, and mark the response as stale.
Step 1 of 6
Write the key from every input that changes the response: path, query, tenant, locale, permission scope, model or prompt version.
Figure. The mechanism in this page's own steps. The figure adds sequence, not new claims.
What to notice. A step can be read alone, but the order is the mechanism. The control lets you stop on any one.
Text version
The page's steps: 1. Write the key from every input that changes the response: path, query, tenant, locale, pe… 2. Choose a freshness policy per object: a lifetime for tolerable staleness, validators for … 3. Set the lifetime from how fast the source changes, and record the choice next to the cach… 4. Invalidate by version rather than by deleting keys, so a deploy changes the key space and… 5. Add request coalescing or a short lock so a popular expired key does not cause a stampede. 6. Serve stale on origin failure only if the product accepts it, and mark the response as st…
- Write the key from every input that changes the response: path, query, tenant, locale, permission scope, model or prompt version.
- Choose a freshness policy per object: a lifetime for tolerable staleness, validators for revalidation, explicit purge for correctness-critical data.
- Set the lifetime from how fast the source changes, and record the choice next to the cache definition.
- Invalidate by version rather than by deleting keys, so a deploy changes the key space and old entries age out.
- Add request coalescing or a short lock so a popular expired key does not cause a stampede.
- Serve stale on origin failure only if the product accepts it, and mark the response as stale.
Trade-offs
| Choice | Good for | Cost | Fails when |
|---|---|---|---|
| Short lifetime | Bounded staleness, simple | More origin load | The origin cannot absorb the miss rate |
| Explicit purge | Precise freshness | Every writer must remember to purge | Writes happen in many services |
| Version in the key | Atomic invalidation, no purge call | Old entries linger until eviction | Storage is constrained |
| Cache everything | Highest hit rate | Stale and sensitive data risks | Responses are personalised |
Good for
Bounded staleness, simple
Cost
More origin load
Fails when
The origin cannot absorb the miss rate
- The origin cannot absorb the miss rate
- Writes happen in many services
- Storage is constrained
- Responses are personalised
Figure. Each choice with what it buys, what it costs, and the case where it loses.
What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.
Text version
Four trade-offs: Short lifetime is good for Bounded staleness, simple, costs More origin load, and fails when The origin cannot absorb the miss rate. Explicit purge is good for Precise freshness, costs Every writer must remember to purge, and fails when Writes happen in many services. Version in the key is good for Atomic invalidation, no purge call, costs Old entries linger until eviction, and fails when Storage is constrained. Cache everything is good for Highest hit rate, costs Stale and sensitive data risks, and fails when Responses are personalised
Failure modes
- Key missing a dimension. A user sees another locale's or tenant's content because the key omitted it. Debug: log the computed key and assert it changes when tenant, locale or permission scope changes.
- Stampede on expiry. A hot key expires and the origin receives a burst of identical misses. Debug: chart miss rate against origin latency for a single key and look for a synchronised spike.
- Purge that never fires. A writer path bypasses the invalidation call and stale data persists indefinitely. Debug: instrument writes and purges, and alert when a write has no corresponding purge.
In production
Track hit rate, age of served entries and origin load per key family. Alert on a sudden hit-rate drop, which usually means a key changed shape, and cap entry size so one object cannot evict the working set.
Common misconceptions
People think: Invalidation is only hard in distributed systems.
Actually: It is hard wherever more than one writer can change the source, which includes a single service with several code paths.
People think: A short lifetime removes the need for invalidation.
Actually: It bounds staleness but does not remove it, and the shorter the lifetime the more load the origin must absorb.
Interview angle
Begin with the key and the staleness budget, then choose between lifetime, validation and purge. Name the stampede and the writer that forgets to purge. Setting a cache header and moving on is not an invalidation design.
Sources
- RFC 9111, HTTP Caching — freshness, validators and cache key rules.
- MDN, HTTP caching — practical Cache-Control and validation behaviour.