Skip to content
AI Engineer Guide.

concept · basics · intermediate · concept 16 of 24

Idempotency

0 of 24 complete

Why it matters

Retries are mandatory. Networks drop responses, load balancers restart, clients time out after the server has already committed. Every one of those cases leaves the caller unsure whether the work happened, and the safe response is to ask again.

Without idempotency, "ask again" is a new operation. The customer is charged twice, two onboarding emails arrive, two tickets are filed, the inventory decrements twice. This is not an edge case at scale: it is a certainty, and it is the reason payment APIs expose idempotency keys as a first-class feature.

The problem gets sharper with agents. A model-driven loop may call the same tool repeatedly after a failed observation — resend the email, re-open the issue, re-run the deployment. The loop cannot know whether the first call landed, so the tool has to make the answer safe.

Mental model

A coat check ticket. Hand over the same ticket twice and you get the same coat, not a second coat. The ticket identifies an operation that has already happened, and the counter's records decide the answer.

The analogy breaks under concurrency. Two people can present the same ticket at two windows at the same instant. A real coat check relies on one attendant and one rack. Software needs an atomic claim — a uniqueness constraint or compare-and-set — or both requests will see "not found" and both will fetch a coat.

How it works

Mechanism, step by stepInteractive
  1. 1The client creates a key per logical operation. One key for "submit this order", reused unchanged across every retry of that submission. A UUID generated inside the retry loop defeats the whole design.
  2. 2The server scopes the key. Keys are namespaced by account and endpoint, so one tenant cannot collide with or probe another's keys. Scope is part of the primary key.
  3. 3Claim the key atomically, before the side effect. Insert the key with a unique constraint and record the request fingerprint. If the insert conflicts, this is a duplicate and you handle it explicitly. The claim must commit before the external call, otherwise a crash between the effect and the record leaves no trace.
  4. 4On duplicate, distinguish three states. Completed: replay the stored status and body. In flight: return a conflict or wait briefly, never execute again. Fingerprint mismatch: reject, because the same key with a different body is a client bug, not a retry.
  5. 5Store enough to replay. Response status, body and any identifiers the caller needs. Replaying a 201 with the original resource ID is what makes the retry transparent.
  6. 6Make the underlying effect idempotent too. A key store protects the API boundary; the database write still needs an upsert or a natural key, and the downstream provider may need its own key. Defence in depth, because the key store can fail after the effect succeeds.

Step 1 of 6

The client creates a key per logical operation. One key for "submit this order", reused unchanged across every retry of that submission. A UUID generated inside the retry loop defeats the whole design.

Step 1/6 · step 1

Figure. The mechanism in this page's own steps. The figure adds sequence, not new claims.

What to notice. A step can be read alone, but the order is the mechanism. The control lets you stop on any one.

Text version

The page's steps: 1. The client creates a key per logical operation. One key for "submit this order", reused u… 2. The server scopes the key. Keys are namespaced by account and endpoint, so one tenant can… 3. Claim the key atomically, before the side effect. Insert the key with a unique constraint… 4. On duplicate, distinguish three states. Completed: replay the stored status and body. In … 5. Store enough to replay. Response status, body and any identifiers the caller needs. Repla… 6. Make the underlying effect idempotent too. A key store protects the API boundary; the dat…

  1. The client creates a key per logical operation. One key for "submit this order", reused unchanged across every retry of that submission. A UUID generated inside the retry loop defeats the whole design.
  2. The server scopes the key. Keys are namespaced by account and endpoint, so one tenant cannot collide with or probe another's keys. Scope is part of the primary key.
  3. Claim the key atomically, before the side effect. Insert the key with a unique constraint and record the request fingerprint. If the insert conflicts, this is a duplicate and you handle it explicitly. The claim must commit before the external call, otherwise a crash between the effect and the record leaves no trace.
  4. On duplicate, distinguish three states. Completed: replay the stored status and body. In flight: return a conflict or wait briefly, never execute again. Fingerprint mismatch: reject, because the same key with a different body is a client bug, not a retry.
  5. Store enough to replay. Response status, body and any identifiers the caller needs. Replaying a 201 with the original resource ID is what makes the retry transparent.
  6. Make the underlying effect idempotent too. A key store protects the API boundary; the database write still needs an upsert or a natural key, and the downstream provider may need its own key. Defence in depth, because the key store can fail after the effect succeeds.
  7. Expire keys deliberately. Retention must cover the longest plausible retry horizon, including a client that goes offline and resumes. After that, the key is garbage, but the effect must still be idempotent by construction — for example an upsert keyed on a business identifier.
  8. Reconcile when the record is incomplete. If a process dies between the effect and the response write, a background job or a status endpoint resolves the ambiguity rather than leaving the caller to guess.

The store

CREATE TABLE idempotency_keys (
  scope         text        NOT NULL,
  key           text        NOT NULL,
  request_hash  text        NOT NULL,
  state         text        NOT NULL CHECK (state IN ('in_flight', 'done')),
  response_code integer,
  response_body jsonb,
  created_at    timestamptz NOT NULL DEFAULT now(),
  PRIMARY KEY (scope, key)
);
 
-- Claim the key. Zero rows returned means a duplicate.
INSERT INTO idempotency_keys (scope, key, request_hash, state)
VALUES ($1, $2, $3, 'in_flight')
ON CONFLICT (scope, key) DO NOTHING
RETURNING key;

The primary key is the concurrency control: one transaction wins the insert, the other gets no row and takes the duplicate path. A read-then-write check without a constraint lets both requests pass the check.

Idempotent by definition
GET, HEAD, PUT, DELETE, OPTIONS, TRACE
Needs a key
POST endpoints that create or charge
Correctness primitive
Unique constraint on (scope, key)
Three duplicate states
Completed, in flight, fingerprint mismatch
Retention rule
Longer than the longest retry horizon

Trade-offs

ApproachGood forCostFails when
Natural key upsertEntities with a stable business identifierRequires a real natural keyIdentifiers are generated per request
Idempotency-key storePayments, orders, external side effectsA store, a TTL policy and duplicate handlingKeys are generated per attempt or never expire
Request-hash dedupeInternal jobs with deterministic payloadsFalse positives on legitimately repeated requestsTwo identical requests are both intended
At-least-once plus reconciliationBatch pipelines, eventual consistencyDelayed correctness, extra jobThe user waits for an immediate answer
No dedupe, manual cleanupPrototypesDouble effects and support loadMoney or email is involved
Trade-off mapInteractive

Good for

Entities with a stable business identifier

Cost

Requires a real natural key

Fails when

Identifiers are generated per request

  1. Identifiers are generated per request
  2. Keys are generated per attempt or never expire
  3. Two identical requests are both intended
  4. The user waits for an immediate answer
  5. Money or email is involved

Figure. Each choice with what it buys, what it costs, and the case where it loses.

What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.

Text version

Four trade-offs: Natural key upsert is good for Entities with a stable business identifier, costs Requires a real natural key, and fails when Identifiers are generated per request. Idempotency-key store is good for Payments, orders, external side effects, costs A store, a TTL policy and duplicate handling, and fails when Keys are generated per attempt or never expire. Request-hash dedupe is good for Internal jobs with deterministic payloads, costs False positives on legitimately repeated requests, and fails when Two identical requests are both intended. At-least-once plus reconciliation is good for Batch pipelines, eventual consistency, costs Delayed correctness, extra job, and fails when The user waits for an immediate answer. No dedupe, manual cleanup is good for Prototypes, costs Double effects and support load, and fails when Money or email is involved

Failure modes

Key generated inside the retry loop. Every attempt is unique and every duplicate executes. Debug: assert one key per logical operation in the client test, not one per HTTP call.

Read-then-write race. Two concurrent duplicates both find no record and both proceed. Debug: run the same request twice in parallel against staging and count the side effects; a unique constraint is the only reliable fix.

Fingerprint ignored. A client reuses a key for a different payload and receives the first response, silently dropping the second request. Debug: store a hash of the canonicalised body and return a distinct error on mismatch.

Unbounded retention. The key table grows without limit and becomes the largest table in the database. Debug: check row counts and index size monthly; partition by creation date and expire on a schedule.

In production

Monitor duplicate rate, replay rate, fingerprint mismatches and in-flight conflicts. A sudden rise in duplicates usually means a client-side timeout is set too low — the server is fine, the caller gives up early. Keep the key store on the same transactional boundary as the primary write where possible; if the effect is external, pair the store with a reconciliation job that queries the provider for the operation's final state. For agent tools, apply the same discipline per tool call: an email tool needs a dedupe key, a ticket tool needs a natural key, and a read-only tool needs none.

Common misconceptions

People think: Idempotency means the response is identical.

Actually: It means the intended effect is identical. A replayed response is a convenience; the essential guarantee is that the second request does not apply the change twice.

People think: A database transaction gives me idempotency across an external API call.

Actually: The transaction cannot roll back a charge or an email that already left the system. You need a recorded key plus an idempotent or reconcilable external effect.

People think: Exactly-once delivery solves this.

Actually: Exactly-once delivery is not achievable end to end over a network that can lose acknowledgements. Systems achieve effectively-once processing through at-least-once delivery plus deduplication.

Interview angle

The question arrives as "how do you prevent a double charge when the client retries?" and the follow-ups test whether you understand the ordering. Answer with the key lifecycle, the atomic claim before the effect, the three duplicate states, and retention. Then mention the failure that remains: a crash after the effect and before the record, and how reconciliation covers it. That last part is what distinguishes a design from a diagram.

See production questions and retries, timeouts and jitter.

Sources