Idempotency Keys: How One Header Stops a Retry From Charging a Customer Twice
A client sends a POST to create a payment. The server charges the card, writes the record, and starts sending the response. Then the connection drops. The client sees a timeout and has no way to tell whether the charge happened. If it retries, the customer may pay twice. If it doesn’t, the order may never complete.
Every API that creates or changes something over a network has this problem. GET, PUT and DELETE are idempotent by definition in HTTP, so repeating them is safe. POST and PATCH aren’t. The fix most payment APIs settled on is a client-generated key sent with the request, which the server uses to recognize a retry and hand back the original answer instead of doing the work again.
How the contract works
The client creates a unique key for one logical operation, usually a random UUID, and sends it in an Idempotency-Key header. The important word is logical. The key belongs to “pay for order 1042”, not to one attempt at the HTTP call. Generate it once, store it next to the pending operation, and send the same key on every retry.
The server keeps a record per key. On the first request it does the work and saves the result. On a later request with the same key it skips the work and returns the saved result. Stripe, the best-known example, saves the status code and body of the first request once execution starts and replays it for later retries, 500 errors included. Keys are up to 255 characters, and Stripe may prune them once they are at least 24 hours old. A key reused after pruning is treated as a new request.
There are three edge cases, and the IETF draft for the header, draft-ietf-httpapi-idempotency-key-header, gives each one a status code:
- No key on an endpoint that requires one:
400 Bad Request. - Same key, different payload:
422 Unprocessable Content. This is a client bug, so it should fail loudly. Servers detect it by storing a fingerprint, such as a hash of the request body, next to the key. - Same key while the first request is still running:
409 Conflict. The client should wait and retry. It doesn’t need to change anything.
The draft defines the header as a Structured Field string, Idempotency-Key: "8e03978e-40d5-43e8-bc93-6894a57f9324", and asks servers to publish their expiry policy. One caveat: the draft expired in April 2026 without becoming an RFC. The header is a de facto standard in payments, not a ratified one, so read each provider’s docs for its exact rules.
A server-side version in about 30 lines
Here’s the whole mechanism in Python and SQLite. The primary key on (account, key) does the real work: the database lets exactly one insert win, so two concurrent requests with the same key can’t both run.
import hashlib, json, sqlite3
db = sqlite3.connect("api.db", isolation_level=None)
db.execute("""CREATE TABLE IF NOT EXISTS idempotency (
account TEXT, key TEXT, fingerprint TEXT,
status INTEGER, body TEXT, -- NULL while the first request runs
created_at TEXT DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (account, key))""")
def fingerprint(payload):
return hashlib.sha256(json.dumps(payload, sort_keys=True).encode()).hexdigest()
def handle(account, key, payload, do_work):
if not key:
return 400, {"error": "Idempotency-Key header required"}
fp = fingerprint(payload)
try:
db.execute("INSERT INTO idempotency (account, key, fingerprint) VALUES (?, ?, ?)",
(account, key, fp))
except sqlite3.IntegrityError: # key seen before
row = db.execute("SELECT fingerprint, status, body FROM idempotency "
"WHERE account = ? AND key = ?", (account, key)).fetchone()
if row[0] != fp:
return 422, {"error": "key reused with a different payload"}
if row[1] is None:
return 409, {"error": "original request still in progress"}
return row[1], json.loads(row[2]) # replay the saved response
status, body = do_work(payload) # runs once per key
db.execute("UPDATE idempotency SET status = ?, body = ? WHERE account = ? AND key = ?",
(status, json.dumps(body), account, key))
return status, body
Run it against a fake charge function and you get one charge for two identical requests, a 422 when the amount changes under the same key, a 409 when the first request hasn’t finished, and a separate charge when a different account happens to use the same key string.
Where implementations go wrong
Keys scoped globally. Scope keys to the account or API credential. Otherwise one customer’s random string can collide with another’s, and somebody gets a stranger’s response.
Keys generated per attempt. A client that makes a fresh UUID inside its retry loop has idempotency in name only. Every retry is a new operation.
A crash leaves the key locked. In the code above, if do_work dies after the insert, the row stays empty and every retry gets 409 forever. Give in-progress rows a timeout after which they can be reclaimed, or delete the row when the work fails before any side effect happened.
Replaying errors you should retry. Saving and replaying a 500 is safe, since the client sees the same failure every time, but it means a transient error is permanent for that key. Stripe’s v1 API does exactly that. Its v2 API instead finishes failed or partly failed work on retry where it can. Decide which behavior you want and document it.
Side effects outside your database. If the operation calls a payment processor, pass an idempotency key downstream too, for example your key plus a suffix for each step. Your own table only protects what your own table controls.
Fingerprints that are too strict. Hash the fields that define the operation, not headers or timestamps a client library adds on its own. A retry that differs only in a User-Agent string shouldn’t get a 422.
Idempotency keys cost a table, an insert and a lookup. Leaving them out costs a double charge and a support ticket that starts with an angry customer, and a timeout is bound to happen eventually.
For how retries and versioning interact with what your API has already promised, see every accident in your API becomes a contract and HTTP status codes used wrong.