@setu-ts/idempotency-plugin

A repeated HTTP request, queue job or broker message is recognised by its key: a repeat of completed work is answered from its record or skipped instead of running again. One core state machine — claim(key, fingerprint, lease) → claimed with a token, then complete(token, record) or release(token) — sits behind one store port in @setu-ts/common, with an in-process store and two cross-replica stores (Redis and a Cloudflare Durable Object). Two entry points reach it: a route-level idempotent(options) middleware plus an @Idempotent() decorator for HTTP, and an ingress behaviour idempotentIngress(options) for queue jobs and broker messages.

The guarantee this package delivers is no duplicate processing within the limits of the store — it is not, and does not claim to be, a single-execution guarantee. It is the first of three guarantees; the other two are out of scope here:

1. No duplicate processing. A second arrival of a key whose work completed is answered from the record (HTTP) or skipped (ingress); a concurrent second arrival is refused (409 on HTTP, a retryable rejection on ingress).
2. The work and its record committed together — tier C (within) delivers this for database work: the claim, the business writes and the record commit in ONE transaction. Side effects outside that transaction are still at-least-once (guarantee 3).
3. External side effects once — achievable only by forwarding a derived key (see derivedIdempotencyKey) to a provider that de-duplicates.

Installation

npm install @setu-ts/idempotency-plugin
# Deno: deno add npm:@setu-ts/idempotency-plugin

The Redis store lazy-loads the optional peer ioredis:

npm install ioredis
# Deno: deno add npm:ioredis

Without it, startup fails with OptionalDependencyUnavailableError naming ioredis.

To use node-redis instead, inject it through the client arm — { type: 'redis', namespace, client: fromNodeRedis(createClient({ url })) } (fromNodeRedis from @setu-ts/common, createClient from redis) — and ioredis is not needed. The store never closes an injected client: call its quit() when your application stops. See Using node-redis.

Usage

import { createApplication } from '@setu-ts/kernel';
import { RuntimePlugin } from '@setu-ts/runtime';
import { IdempotencyPlugin, idempotent } from '@setu-ts/idempotency-plugin';

const app = createApplication({
  plugins: [RuntimePlugin(), IdempotencyPlugin({ store: { type: 'memory' } })],
});

app.router.post('/payments', {
  middleware: [idempotent()],
  handler: (ctx) => ctx.response.status(201).json({ id: 'pay-1' }),
});

await app.start({ port: 3000 });

Placement

List idempotent() LAST in the route’s middleware array — after guards and validation. An unkeyed or invalid request that never reaches the store must not consume a key, and a request that fails validation must be retryable with the same key. @Idempotent() is appended last by the decorator plugin for the same reason.

The HTTP check order

The middleware runs these steps in this order:

1. Safe method — GET, HEAD, OPTIONS pass through with no store call.
2. Key extraction — from the configured header (default Idempotency-Key), a body field, or a function.
3. Missing key with required: false passes through, before any principal check.
4. Principal — principal: 'required' (the default) answers 401 when no principal is present. With principal: 'optional', every anonymous request shares one scope, so an anonymous caller who sends another anonymous caller’s key is served that caller’s stored response. Pair it with response: 'status' unless the response is safe to show any anonymous caller.
5. Missing key with required: true answers 400; an invalid key answers 400.
6. Fingerprint — by default a SHA-256 over the method, path, raw query, content-type and body bytes.
7. Scope and key derivation, then claim.
8. Outcome — replay, or an error status.

Failure classification

After a claim, the middleware runs the handler and then:

• a thrown error — including a thrown 4xx — releases the claim and rethrows the original error;
• a returned 2xx/4xx (except 408, 425, 429) is recorded;
• a returned 5xx, 408, 425, 429, or any status below 200 is released, not recorded. This is a deliberate departure from providers that replay 500s.

The caller sees the response the handler produced, even if complete or release failed. After a failed complete the record stays in-progress: a retry inside the lease gets 409, a retry after it re-executes.

Replay rules

A replay writes the stored status and headers, adds Idempotent-Replayed: true, and sends the stored body. Only an ALLOW list of headers is stored and replayed; a fixed DENY list (Set-Cookie, Content-Language, RateLimit-*, Retry-After, X-Request-Id, traceparent, Date, Content-Length, …) is never stored and is refused in replayHeaders. The stored record is re-validated on every replay (status 200–499, current allow/deny lists, body size), so a tampered record answers 503 instead of being replayed.

Query order matters. The default fingerprint hashes the raw query exactly as received, so ?a=1&b=2 and ?b=2&a=1 fingerprint differently.

From a browser

A keyed request carries a non-safelisted header, so a cross-origin browser call is preflighted. The corsMiddleware from @setu-ts/http-security-plugin must allow it and expose the replay header: allowedHeaders must include idempotency-key, and exposedHeaders: ['idempotent-replayed'] is needed for the page to read the Idempotent-Replayed header. The CORS middleware must run before idempotent(): a replay only adds headers, so the exposed list survives it.

Send the key per call through the generated operation’s trailing request parameter (request?: OperationRequestOptions): api.createOrder({ body }, { idempotencyKey }). ClientOptions.idempotency (ClientIdempotencyOptions) generates a key by default for POST and PATCH; ClientOptions.headers carrying the key header is refused while it is configured, so the per-call idempotencyKey is the way to reuse one key across SDK client instances acting for the same principal (a retry after a page reload, say). Records are scoped by the authenticated identity, so on a route with principal: 'required' (the default) another principal’s request with the same key runs its own work rather than replaying yours. That isolation holds only for authenticated callers and a correctly built scope: on a principal: 'optional' route every anonymous caller shares one scope, so a caller who guesses another anonymous caller’s key is served that caller’s stored result (see the HTTP check order). Generate keys with enough entropy to be unguessable, and pair such a route with response: 'status' unless its response is safe to show any anonymous caller.

A custom key header is renamed in three places: the route’s key: { header }, ClientIdempotencyOptions.header on the SDK side, and the CORS allowedHeaders entry (the preflight must allow the header the SDK sends under its new name).

Ingress: the allow-list and the consumer identity

idempotentIngress(options) acts only on an explicit topics (broker) and/or jobNames (queue) allow-list. Everything else — including every scheduler and websocket envelope, and the realtime backplane’s topic — passes through untouched, with no store call.

The key includes the consumer identity (IngressContext.consumer), so two subscribers on one topic each run once rather than one being skipped as a “duplicate”. On a listed topic, pass queue to subscribe for competing-consumer subscriptions; a queue-less subscription’s consumer id is per process, so a redelivery to another replica is not de-duplicated.

On a listed topic or job with no key, the behaviour throws a named refusal: a queue job dead-letters after its attempts, and a broker redelivers per its own policy.

Store guarantees

Tier A on Cloudflare Workers is per isolate — use the Durable Object store there. The ingress entry point is not available on Workers in this release.

Tier C: the work and its record commit together

within(options, fn) is tier C, for a handler that owns a database transaction. It opens ONE transaction, creates a claim row in it FIRST, runs fn with that transaction’s unit of work, creates a result row holding fn’s JSON result, and commits — so the work and its idempotency record are all-or-nothing. A repeated key whose record is committed returns the stored result without running fn (replayed: true); a concurrent duplicate loses the race on the claim’s primary key and its business writes roll back with it, then it replays the winner’s result — or, where the backend reports a write conflict before the winner committed, rejects with a retryable 409 (which @setu-ts/sdk’s keyed client retries).

On both the first call and a replay, value is the JSON round trip of what fn returned: a Date becomes an ISO string, undefined object members are absent, and nested toJSON() methods determine their stored representation. A void result remains undefined.

scope is REQUIRED and must be built ONLY from authenticated identity (typically JSON.stringify([tenantId, principalId])); '' declares the record global on purpose. Do not join parts with a separator that can appear inside them: distinct identities can collide. The raw key, scope and namespace are never stored — only their derived hashes.

Configure it with IdempotencyPlugin({ transactional: { store } }), where store is createDatabaseIdempotencyStore() from @setu-ts/database-plugin (or any ITransactionalIdempotencyStore). The store is resolved and verify()d at onInit.

The default purge is scheduled, so the example registers SchedulerPlugin(). Without CAPABILITIES.SCHEDULER, startup rejects with IdempotencyConfigurationError. Alternatively set transactional.purge: { schedule: false } and call idempotency.purgeTransactional() yourself. That alternative is mandatory on Workers, where SchedulerPlugin throws; call the purge from a Cron Trigger.

A handler that creates an order calls within itself, because middleware cannot own the handler’s transaction. The scope comes from the authenticated principal and tenant only — never from the request body — and is encoded with JSON.stringify, so no tenant or user id can be crafted to collide with another’s:

import { CAPABILITIES } from '@setu-ts/common';
import type { IIdempotencyService } from '@setu-ts/common';
import { createDatabaseIdempotencyStore, DatabasePlugin } from '@setu-ts/database-plugin';
import type { IUnitOfWork } from '@setu-ts/database-plugin';
import { SchedulerPlugin } from '@setu-ts/scheduler-plugin';
import { IdempotencyPlugin } from '@setu-ts/idempotency-plugin';
import { createApplication } from '@setu-ts/kernel';
import { RuntimePlugin } from '@setu-ts/runtime';

const app = createApplication({
  plugins: [
    RuntimePlugin(),
    DatabasePlugin(),
    SchedulerPlugin(),
    IdempotencyPlugin({ transactional: { store: createDatabaseIdempotencyStore() } }),
  ],
});

app.router.post('/orders', async (ctx) => {
  const key = ctx.request.headers.get('Idempotency-Key');
  const principal = ctx.request.user?.id;
  if (key === null || principal === undefined) {
    return ctx.response.status(400).json({ error: 'an authenticated, keyed request is required' });
  }
  const body = await ctx.request.json<{ item: string }>();
  const idempotency = ctx.services.get<IIdempotencyService>(CAPABILITIES.IDEMPOTENCY);
  const { value, replayed } = await idempotency.within<{ id: string }, IUnitOfWork>(
    {
      key,
      namespace: 'orders.create',
      scope: JSON.stringify([ctx.request.tenant?.id ?? null, principal]),
      fingerprint: body,
    },
    async (uow) => {
      const order = { id: crypto.randomUUID(), item: body.item };
      await uow.getRepository('Order').create(order);
      return { id: order.id };
    },
  );
  return ctx.response.status(replayed ? 200 : 201).json(value);
});

Not verified: DynamoDB’s overlapping-service TransactionConflictException outcome on DynamoDB Local (M90f). Tier C was driven against a real Prisma 7.10 client on PostgreSQL once, by hand; the committed real-backend suite uses Drizzle.

Expiry and retention. A record is authoritative from commit until the scheduled purge deletes it; expiresAt (from within’s ttlMs, default 86,400,000 ms) only makes it ELIGIBLE. within’s ttlMs must be an integer from 60,000 ms (one minute) to 2,592,000,000 ms (30 days); outside that range every call fails with IdempotencyConfigurationError “idempotency: within ttlMs is out of range”. The purge deletes up to purge.batch (default 100) eligible records every purge.intervalMs (default 60,000), so the table stays bounded only while inflow stays below that rate. A key reused after ttlMs replays until the purge runs. On Workers, set purge: { schedule: false } and call service.purgeTransactional() from a Cron Trigger.

Known purge race. Two purges running at once (two replicas, or a manual purgeTransactional() beside the scheduled one) can delete a record that a client re-created in the moment between them, so a retry of that request runs fn again. The purge re-reads each record inside its delete transaction, which closes the window on MongoDB and narrows it elsewhere to about one round trip on PostgreSQL (READ COMMITTED) and to the commit of the deferred batch on DynamoDB and D1. The portable repository does offer a conditional delete (deleteWhere(id, where), M105), but the purge does not use it: a record is two rows (id and ${id}.r) that must go together, and deferred-write transaction sources omit conditional members. Run the scheduled purge on one replica if a key can be reused right after it expires.

Memory store capacity

The memory store caps each scope at maxEntriesPerScope (default 1,000) and the whole store at its global entry and byte caps. A scope is the tenant and principal a key is derived for. A caller over its scope’s cap gets 429 while callers in other scopes still claim, but only until the GLOBAL cap fills, after which every new claim gets 503. One principal can occupy several scopes when the tenant is chosen by the client (for example a tenant read from a request header), so the per-scope cap alone does not stop one caller filling the store. Rate-limit upstream, or resolve the tenant from a verified source.

Redis durability caveat and version floor

A completed Redis record survives only under maxmemory-policy noeviction and persistence that does not drop acknowledged writes on failover. Under any eviction policy, or when an asynchronous replica is promoted after a primary dies, a completed record can vanish and the next duplicate executes. The store warns at connect time when maxmemory-policy is not noeviction. Redis ≥ 5 is required: TIME inside a script is safe only under script-effects replication, the default from Redis 5.

The queue retry span versus the lease

A lease held by a crashed holder blocks redeliveries of its key until it lapses; each redelivery inside the lease is refused in-progress, which consumes a queue attempt. The retry span (the sum of computeBackoffMs(2..maxAttempts)) must EXCEED the lease: with the default backoff and the default 30 s ingress lease that means defaultMaxAttempts ≥ 6. The queue’s retry configuration is not readable from this plugin, so this is documented, not checked.

Principal identity

A principal’s identity is its IPrincipal.id alone — the type carries no issuer or strategy member. An application authenticating through more than one issuer MUST ensure principal ids are unique across them (for example by prefixing them in the strategy); otherwise two issuers’ identical subjects share a scope.

Multi-replica deployments

Use the Redis or Durable Object store, each with a required, per-application namespace so two applications sharing a backend never read each other’s records.

Retention and personal data

Completed records live for ttlMs (default 24 h). A stored response may hold personal data and secrets at rest for that time, so retention counts toward the data inventory and erasure timelines. Use response: 'status' and/or redaction where a body must not be stored. Note that in status mode the fingerprint — an unsalted SHA-256 of the request body — is still stored, so a low-entropy body (a short code, a PIN) can be confirmed by brute force from it.

Forwarding a key to a provider

import { derivedIdempotencyKey, idempotent } from '@setu-ts/idempotency-plugin';

app.router.post('/charges', {
  middleware: [idempotent()],
  handler: (ctx) => {
    // `key` is stable per principal + namespace + client key; forward it to a
    // provider that de-duplicates.
    const key = derivedIdempotencyKey(ctx) ?? '';
    return ctx.response.status(201).json({ forwardedKey: key });
  },
});

What this mechanism is not

• It is not a substitute for a business uniqueness constraint (“one payroll run per company and period”) — that is a unique index, not an idempotency key.
• It does not de-duplicate scheduler ticks; those are already slot-locked.
• It does not make an outbound call happen once, unless the provider de-duplicates on the key you forward.

The IETF draft

The Idempotency-Key header and the status codes follow draft-ietf-httpapi-idempotency-key-header, draft-07, expired. It is cited for the header field and the 400/409/422 semantics this package implements.

Exports

Generated from the package barrel by pnpm run docs:exports; pnpm run check:docs fails when it drifts.

Full API

Every export and option is documented in PUBLIC_API.md.

API reference

IdempotencyPlugin() (@setu-ts/idempotency-plugin)

A repeated HTTP request, queue job or broker message is recognised by its key, and a repeat of completed work is answered from its record or skipped instead of running again, over one claim/complete/release state machine and an in-process, Redis or Cloudflare Durable Object store. The guarantee is no duplicate processing within the limits of the store — it is not a single-execution guarantee. Tier C (within, M109b) extends this to plain code: the work and its record commit in ONE database transaction, so a lost race rolls the business writes back with the claim. See packages/idempotency-plugin/README.md for placement, the HTTP check order, failure classification, replay rules, the ingress allow-list, the tier-C backend table and the store guarantees.

within returns the JSON round trip of what fn returned on both the first call and a replay. Dates become ISO strings, undefined object members are absent, and nested toJSON() output determines the stored value. A void result returns undefined on both paths.

Values (runtime exports)

Types

The queue retry span versus the lease

A crashed holder’s claim blocks redeliveries of its key until the lease lapses, and each redelivery inside the lease consumes a queue attempt. The retry span (the sum of computeBackoffMs(2..maxAttempts)) must EXCEED the lease: with the default backoff and the default 30 s ingress lease that means defaultMaxAttempts ≥ 6. The queue’s retry configuration is not readable from this plugin, so this is documented, not checked.