A 24-hour idempotency key covers 3.3% of the replay window

Twenty-four hours is the retention I see most often on an idempotency table, and it comes from a sentence that was never about your storage. The sentence is Stripe’s — it says how long Stripe keeps the key on its own side. That same vendor will replay a webhook at you four weeks later.

An idempotency key belongs in the same transaction as the effect it protects, and it lives for as long as the counterparty is still allowed to send that request again. For a Stripe webhook that is 30 days, not the 24 hours printed in the API reference. Two different numbers, two different jobs.

Where does an idempotency key live?

An idempotency key lives in the same transaction as the effect it protects: same database, same commit. The guard is a unique index on the key column. Not a check in application code. A check reads, decides, then writes, so two concurrent replays both read nothing and both proceed. The index refuses the second insert.

Put it anywhere else and you buy the problem back. A cache in front of the database leaves two systems that have to agree whether an effect happened, and a window where they don’t. Every payments build we take on under software development starts with that index.

The specification is no help. Its silence is the interesting part. RFC 9110 Section 9.2.2 defines idempotency as a property of the request method, and the paragraph licensing this pattern reads:

A client SHOULD NOT automatically retry a request with a non-idempotent method unless it has some means to know that the request semantics are actually idempotent, regardless of the method, or some means to detect that the original request was never applied.

The key is that means to know. But Section 9.2.2 ends at line 3904 of a 10,785-line document and never says where that means is kept, or for how long. Grep the plain text for storage: six lines, none about idempotency state. You inherit no retention rule, so you copy a vendor’s.

How long to keep a key: read the counterparty’s retry schedule

Keep a key for exactly as long as the counterparty is still allowed to send the same request again, which is a number they publish rather than a round number you pick. The derivation takes a minute and nobody argues with it once written down. I’ve almost never seen it done on a live codebase, which is a claim about the codebases I’ve opened rather than about the literature. Teams agree on the key, the index and the conflict response, then set a TTL by feel.

Two directions, and they don’t share a number. Outbound, the provider owns the record and their retention caps how long your retries mean anything; inbound, you own the record and their replay horizon is what yours has to survive. One number for both is the mistake, nearly always the outbound one applied inbound.

The standards track refuses to pick a duration for you. Section 2.3 of draft-ietf-httpapi-idempotency-key-header-08, published on 22 October 2026, puts it in two sentences:

The resource MAY require time based idempotency keys to be able to purge or delete a key upon its expiry. The resource SHOULD define such expiration policy and publish it in the documentation.

Section 2.7 defines errors for a missing key, a reused key and an outstanding request — and nothing for an expired one. An expired key doesn’t fail; it succeeds, for the second time.

Stripe keeps keys for at least 24 hours and retries webhooks for three days

Stripe keeps an API v1 idempotency key for at least 24 hours and retries a live-mode webhook for up to three days, and the two numbers govern different things. The retention sentence, from Stripe’s advanced error handling page, read on 29 October 2026:

Stripe retains keys for at least 24 hours and can remove them after that. Reusing a key after it’s removed can execute a new request.

Read “at least” slowly — it is a floor, not a deadline. The same section caps your own retrying at 24 hours too, then tells you to stop: “If the outcome is still unknown after that window, don’t retry with either the old key or a new one.” Stripe doesn’t say why. My reading is that their copy of the key might be gone by then.

Inbound is a different set of numbers. Stripe’s webhook documentation says it “attempts to deliver events to your destination for up to three days with an exponential back off in live mode”. Replay after that is manual and still legitimate: the Dashboard’s Resend button works for 15 days, the CLI for 30.

And then the number that undoes the 24-hour habit. Idempotency differences between API v1 and API v2, near the foot of the API v2 overview, prints the two windows as parallel bullets: v1 “if they use the same idempotency key and occur within 24 hours of each other”, v2 “if they use the same idempotency key, are made to the same API, occur within the scope of the same account or sandbox, and occur within 30 days of each other”. Those extra clauses are the trade: v2 buys its month by matching on more than the key.

Stripe’s own windows, each figure quoted from the pages linked above and read on 29 October 2026:

Two other vendors widen the spread. PayPal’s idempotency reference gives none: “for information about how long the server stores the ID, see the reference for your API”. The only figure I found, 45 days, sits in the API requests page inside a worked example about refunding a capture. Adyen publishes a range: “Idempotency keys are valid for a period of 7 to 14 days after first submission.” I wouldn’t design against a worked example.

Keep an idempotency key for 35 days, not 24 — the replay floor

The replay floor is the rule we work to: an idempotency record has to outlive the longest replay horizon the counterparty publishes, plus your own longest planned outage. Set the tolerance first. Ours is zero legitimate replays applied twice — not rare, not low, zero, because a double-applied ledger entry gets found weeks later by reconciliation rather than by an alert. Then read the distribution: 24 hours to 30 days inside Stripe alone, 7 to 14 days at Adyen, 45 in PayPal’s one example. Then take the longest that can legitimately reach you, which inbound from Stripe is the 30-day CLI resend. Add your own margin. Mine is five days, a long weekend plus a restore, and it could be shorter if you can restore inside a day. 30 + 5 = 35.

Same vendor, same Idempotency-Key header, two API versions, two numbers 30 times apart.

What happens when the key expires before the retry arrives

The retry is treated as a new request, the effect is applied a second time, and no error is raised anywhere. Ferrystone is the worked example, and it is invented: a payments ledger this library keeps on hand so a failure sequence has somewhere concrete to happen. Nobody’s real system is described below. Its handler stores processed Stripe event IDs with a 24-hour TTL copied from the API reference. A payment_intent.succeeded arrives at hour 0, the handler commits the ledger row, and the pod is OOM-killed before it returns a 200. Stripe records a failed delivery. The deploy is in a crash loop, so every retry through the next day dies at the load balancer and nothing logs an error. Ferrystone rolls back at hour 29 and the next retry lands at hour 30. Both are chosen; all they have to do is sit between Ferrystone’s 24 and Stripe’s 72. The record expired 30 − 24 = 6 hours earlier, so the lookup misses, the handler writes the entry a second time, and Stripe marks the event delivered.

Nothing in that sequence is a bug in anybody’s code. The only wrong decision was a number.

Percentages make it harder to argue with. Stripe’s automatic window is 72 hours, so a 24-hour store covers 24 ÷ 72 = 33.3% of it. The resend horizon is 720 hours, so the same store covers 24 ÷ 720 = 3.33% and leaves 696 hours, or 96.7%, with nothing guarding it. A manual resend “doesn’t cancel Stripe’s automatic retry behavior”, so both paths can land, and Stripe’s advice on duplicate events is to log the event IDs you have processed without ever saying for how long. The companion piece on what to store and what to return when a payment request arrives twice carries the rest: which response body you keep, which status code a replay gets, and what to do when the same key turns up with a different payload.

How much storage 30 days of idempotency keys costs

Thirty days of idempotency keys costs 6.84 GB in PostgreSQL 18 at twenty writes a second. Every figure comes from the PostgreSQL documentation, one struct in its source tree, and two numbers I chose and label as chosen.

create table payment_idempotency (
  key          uuid        not null,
  request_hash bytea       not null,
  state        smallint    not null,
  status_code  smallint    not null,
  created_at   timestamptz not null
) partition by range (created_at);

PostgreSQL 18’s Section 66.6.1, Table Row Layout, says there is “a fixed-size header (occupying 23 bytes on most machines), followed by an optional null bitmap…”, and the section above it gives item identifiers four bytes each. Twenty-three rounds to 24 on MAXALIGN, which is 8 on x86-64. Then the columns, in declaration order: key is 16 bytes and ends at offset 40. The obvious next step is wrong. A 32-byte digest doesn’t sit behind the four-byte varlena header: Section 66.2, on TOAST, says a value shorter than 127 bytes gets “only a single-byte header instead of the normal four-byte header” and that such values “aren’t aligned on any particular boundary”. So request_hash is 33 bytes and ends at 73; the smallints take a pad byte and run to 77; created_at needs an eight-byte boundary, so two more pad bytes put it at 80. 24 + 16 + 33 + 1 + 2 + 2 + 2 + 8 = 88, plus the line pointer: 92 bytes of heap. Reading the TOAST section changed the parts, not the total. The three bytes the short header saves come straight back as alignment padding.

A unique btree on key adds 8 + 16 + 4 = 28 bytes: an index tuple header, the uuid, a line pointer. That 8 is in the source rather than the manual: IndexTupleData in src/include/access/itup.h is a six-byte ItemPointerData plus a two-byte t_info. Random UUID v4 inserts don’t pack leaf pages tight. I work on 70% fill, which gives 28 ÷ 0.7 = 40 bytes, and that 70% is a working assumption I cannot source to the documentation. Heap plus index is 92 + 40 = 132 bytes per key.

Twenty writes a second is chosen, not measured, and picked so the division stays legible: 20 × 86,400 = 1,728,000 keys a day. At 132 bytes each that is 1,728,000 × 132 = 228,096,000 bytes, or 228 MB a day. Thirty days is 228,096,000 × 30 = 6,842,880,000 bytes, or 6.84 GB. Thirty-five days, the replay floor, is 1,728,000 × 35 = 60,480,000 rows at 132 bytes, which is 7,983,360,000 bytes, or 7.98 GB. Derive it again from the other end: 86,400 × 132 = 11,404,800 bytes, or 11.4 MB per request-per-second-day, and 20 × 30 = 600 of those × 11.4 MB = 6,840 MB. Both routes agree.

Double the chosen rate and the shape holds: 40 × 86,400 × 132 = 456,192,000 bytes a day, so 13.7 GB at 30 days. Now derate the row four times, 132 bytes to 528, for a text key and index bloat: 1,728,000 × 30 = 51,840,000 rows at 528 bytes = 27,371,520,000 bytes, or 27.4 GB. The decision does not flip. Going from a 24-hour store to the 30-day window costs 6.842 − 0.228 = 6.614 GB and buys the other 96.7%; going on to the 35-day floor costs 7.983 − 0.228 = 7.755 GB and buys (35 × 24 − 24) ÷ 840 = 816 ÷ 840 = 97.1%.

Is Redis a safe store for idempotency keys?

No, and the reason is the access pattern rather than durability. An idempotency key is written once and read at most once, so under allkeys-lru the oldest records in the instance go first, and the oldest record is the one guarding the far end of the replay window.

Redis recommends that policy with a condition attached, and the condition is the argument. The key eviction page says to “Use allkeys-lru when you expect that a subset of elements will be accessed far more often than the rest”, because that is “a very common case according to the Pareto principle”. An idempotency store has no hot subset. Redis stated the precondition; the reader keeps the second half of the sentence, that the policy “is a good default option if you have no reason to prefer any others”, and drops the first. The one-line definition, “Evict the least recently used (LRU) keys”, says nothing about expiry, so a key with 29 days left on its TTL is evictable today.

The open-source default fails differently. redis.conf at tag 7.4.0 carries the directive commented out under the line “The default is:”, and two lines below that sits maxmemory-policy noeviction, described in the same comment block as “Don’t evict anything, just return an error on write operations”. Under memory pressure you either lose records or stop writing them. Eviction is driven by writes rather than by a timer, so it bites hardest during the spike that produces the duplicates.

There is a real case for Redis and it is the in-flight lock. A single round trip claiming a key for the few seconds a request runs turns a concurrent duplicate into a 409 and keeps a hot row out of the money table. Use it for that. The durable receipt belongs in the same commit as the ledger row, because at 132 bytes a key, 35 days of receipts is 7.98 GB — fine on a disk, dear as RAM. I would not sign off on a Redis-only idempotency store for anything that moves money.

Partition by day: deleting expired rows doesn’t shrink the table

Create one partition a day and drop the oldest, because a DELETE marks space reusable without handing it back. PostgreSQL 18’s routine vacuuming chapter is explicit:

The standard form of VACUUM removes dead row versions in tables and indexes and marks the space available for future reuse. However, it will not return the space to the operating system, except in the special case where one or more pages at the end of a table become entirely free…

The storage arithmetic above is wrong for that reason, and the hidden term is autovacuum lag rather than row count. A table on a nightly DELETE has 100% turnover and never settles at 30 days of rows. Steady state is (30 days + however far autovacuum has fallen behind) × 228 MB, and the dead entries sit in the unique index too. I don’t have a defensible number for that lag. I reuse the five days from the replay floor for want of a better one, and it is the figure here I would most want to replace with a measurement. VACUUM FULL would compact it, and the same chapter rules that out of a payment write path: it “requires an ACCESS EXCLUSIVE lock on the table it is working on”. Dropping a partition unlinks a file instead.

Below roughly one idempotent write a second I’d skip the partitioning and run a nightly DELETE, because 11.4 MB a day never bloats into anything worth an hour.

Two things to check before the retention number matters. Write the key in the transaction that starts the work, not the one that finishes it: Stripe’s API reference on idempotent requests says it saves the first response “after endpoint execution begins”, and a key stored only on success leaves the gap between the commit and the response unguarded — the gap Ferrystone fell into at hour 0. Put that record and the ledger row in two databases and you have a distributed transaction, which is where most monolith splits actually fail. And a Stripe 400 means two opposite things depending on when it was raised. One raised before execution begins (invalid parameters is Stripe’s own example) isn’t cached and can be retried once fixed; one raised after is cached, and “Retrying a cached 400 response with the same key returns the same error”. Nothing in the body distinguishes them. The header does, and it is spelt Idempotent-Replayed, not Idempotency-Replayed.