Exactly-Once Delivery Doesn't Exist

A customer gets charged twice. The logs show one payment event, published once. The broker's documentation says "exactly-once." The consumer looks correct. Everyone stares at it for two days.

Nothing is broken. The system is behaving exactly as designed, and the design was based on a guarantee that cannot exist.

Why the network makes it impossible

Strip it to two machines and one message.

The sender transmits. Then it waits for an acknowledgement. The ack does not arrive.

What happened? Either the message never got there, or it got there, was processed, and the ack was lost on the way back. From the sender's position these two worlds are completely indistinguishable. No amount of protocol design fixes that; the information simply is not available on its side of the wire.

So there are two choices, and only two:

• Retry. If the message did arrive, you have now sent it twice → at-least-once, with duplicates.
• Don't retry. If the message did not arrive, it is gone → at-most-once, with loss.

That is the entire menu. "Exactly-once" would require knowing which world you are in, and that knowledge does not exist. This is not a limitation of your broker. It is a property of unreliable channels.

What "guaranteed delivery" actually buys

Guaranteed Delivery is a real and valuable pattern — it is just not the one people think they bought.

It persists each message to durable storage at every hop, so a crash of the sender, broker, or receiver does not lose it, and it keeps retrying until the receiver acknowledges. What you get is durability plus persistence of effort: the message will arrive, eventually, across failures, and you never write retry logic.

You are buying reliability, not uniqueness — and the trade is explicit in the pattern itself, not hidden in the small print: retries mean a message can be delivered more than once, so consumers must be idempotent. Turning on guaranteed delivery does not reduce your duplicate risk — it is the mechanism that creates duplicates, deliberately, in exchange for never losing anything.

Read that once more if your instinct is "we'll enable the reliable mode to be safe." Reliable mode is the duplicate generator.

The exactly-once that vendors do mean

Vendors are not lying, but they are describing something narrower than the phrase suggests.

Kafka-style exactly-once is exactly-once processing within a closed transactional boundary: consume from a topic, update state, produce to another topic, and commit the offset — all atomically. Inside that boundary it is genuine and useful.

It says nothing whatsoever about side effects outside the boundary. Charging a card is not in the transaction. Sending an email is not in the transaction. Calling a third-party API is not in the transaction. The moment your handler touches the outside world, you are back to at-least-once and the guarantee has quietly stopped applying to the part you actually care about.

Build the effect at the receiver

Since the transport cannot give you exactly-once, you produce the effect where the information exists — at the receiver, which can remember what it has already done. That is the Idempotent Receiver.

Two routes, and the first is better whenever it is available.

Make the operation naturally idempotent. State the effect absolutely, not relatively:

// Not idempotent: two deliveries, two increments.
await db.query('UPDATE orders SET attempts = attempts + 1 WHERE id = $1', [id]);

// Idempotent: two deliveries, same final state.
await db.query('UPDATE orders SET status = $2 WHERE id = $1', [id, 'shipped']);

Or deduplicate explicitly, keyed on business identity:

async function handlePaymentRequested(msg: PaymentRequested): Promise<void> {
  await db.transaction(async (tx) => {
    // The dedup row and the side effect commit TOGETHER, or neither does.
    const claimed = await tx.query(
      'INSERT INTO processed_payments (payment_intent_id) VALUES ($1) ON CONFLICT DO NOTHING',
      [msg.paymentIntentId],
    );
    if (claimed.rowCount === 0) return; // already handled — a duplicate

    await tx.query('INSERT INTO ledger (payment_intent_id, amount) VALUES ($1, $2)', [
      msg.paymentIntentId,
      msg.amount,
    ]);
  });
}

Three details in that snippet carry all the weight:

One transaction. If you charge first and record the dedup key afterwards in a separate commit, a crash between them reproduces precisely the bug you are preventing. The record of "I did this" must land atomically with the doing.

A business key, not a message id. If the producer retries, you get a new message id for the same real-world event. Only paymentIntentId catches that. Deduplicating on message id defends against redelivery and nothing else.

ON CONFLICT DO NOTHING as the claim. The uniqueness constraint is the arbiter, so two concurrent consumers racing on the same message resolve correctly at the database rather than in application logic.

When the side effect isn't yours

The awkward case: the effect lives in someone else's system. You cannot put a third-party charge inside your transaction.

Two options, in order of preference. If the provider offers an idempotency key — Stripe-style — use it; you are pushing the same pattern across their boundary and letting them dedupe. If it does not, put your dedup claim in front of the call and accept a genuine, unavoidable risk window: crash after the external call but before recording success, and you will retry it. Narrow that window, monitor it, and reconcile after the fact. You cannot close it, and a design that pretends otherwise is worse than one that plans for it.

Design rules

1. Assume duplicates. Always. Not as a defensive posture — as the documented behaviour of every reliable transport you will use.
2. Key on business identity, not on message id.
3. Commit the dedup record with the side effect, in one transaction.
4. Put the dedup where the side effect is. Deduplicating in a gateway three services upstream protects nothing.
5. Give the dedup store a retention policy, and make the window longer than the broker's maximum retry horizon. Keys kept forever are a slow leak; keys expired early reopen the hole.
6. Remember idempotency is not ordering. A duplicate-safe consumer can still see events out of order. Different problem, separate design.

The takeaway

The question to ask about any consumer you own is not "does our broker guarantee exactly-once?" That question has one answer and it is no.

Ask instead:

When this handler runs twice with the same message, what does the customer see?

If you cannot answer that from the code in front of you, you do not have an exactly-once system. You have an at-least-once system that has not been unlucky yet.