No lost turns: an AI reply pipeline that can’t silently drop a customer

In a customer-messaging product the worst failure isn’t an error. It’s silence.

A customer messages a business on WhatsApp at 11:42 pm. The AI is supposed to answer. Something goes wrong somewhere between the webhook and the reply, and nothing happens. There’s no error page and no stack trace in front of anyone. The customer just never hears back, and the business never knows it lost them.

On ezychat.io, the AI customer-conversation platform I build and run, I call that a lost turn. This post is how the pipeline makes lost turns impossible to lose silently: every owed reply ends as an answer or with a person.

Four ways a turn disappears

The AI reply runs as a Celery task on its own queue. The task is enqueued after the database transaction that saves the inbound message commits. That’s the right order, but it’s best-effort, and the task is acknowledged when a worker picks it up, not when it finishes. Without a backstop, a turn is lost when:

1. The enqueue fails after the commit. Redis is down for a moment, and the message is saved with no task behind it.
2. The broker loses the task. A Redis restart or an eviction drops it.
3. The worker is killed mid-turn. A deploy or an out-of-memory kill ends the process, and the task was already acknowledged.
4. The turn raises and rolls back. An exception anywhere in the turn undoes its work.

Retries help with the fourth, a little. They do nothing for the first three, because there’s no task left to retry. The fix had to start somewhere other than the queue.

Rule 1: the debt is written with the message

When a message makes a reply due, the conversation gets an owed-since timestamp, set in the same database transaction that inserts the message.

That one decision carries most of the design. If the message exists, the obligation exists. It doesn’t matter whether the enqueue happened, whether the broker kept the task, or whether a worker survived. The database, the one component with real durability guarantees, knows a reply is owed and since when.

Voice notes and images work the same way. The obligation is written with the message, before the slower “understand this audio or image” step runs. So a lost transcription is found too, not just a lost reply.

Rule 2: a turn settles exactly what it owed

A turn reads the owed timestamp when it starts. In the same transaction as its outcome (a reply, a handoff to a person, a draft for approval, or a deliberate decision not to answer), it clears the timestamp, but only if it hasn’t changed:

UPDATE conversations
   SET owed_since = NULL
 WHERE id = :conversation_id
   AND owed_since = :owed_since_seen_at_start;

That compare-and-clear handles the race that matters. If the customer sends another message while the turn is running, the new message writes a newer owed timestamp. The running turn can’t clear it, so the newer message gets its own turn. And a turn that raises rolls back with the timestamp still set, which is exactly what we want.

Rule 3: a sweeper that never talks to customers

Every 60 seconds a sweep looks for replies still owed after three minutes. It runs on the messages queue, not the AI queue, so it keeps working when AI work is backed up. That is precisely when it’s needed.

For each owed conversation it walks a short ladder:

• A turn is already running (an in-flight marker with a ten-minute expiry): leave it alone.
• It has been owed for over an hour: hand it to a person. An AI reply that late does more harm than good.
• A reply was already delivered after the turn became owed: hand it to a person. Replaying would message the customer twice. (Rule 4 explains how we know.)
• Three replays haven’t settled it: hand it to a person.
• Otherwise: replay the turn, carrying the owed timestamp as its token.

The token makes replays safe. A replay queued behind a turn that has since finished finds the timestamp changed and exits as already handled, before any model call. Nobody pays for a duplicate LLM call, and nobody gets a duplicate message.

A hand-over uses the same path the pipeline uses when the AI can’t answer. It creates an escalation work item with a plain-language reason, pauses the AI on that conversation, and notifies the staff. The sweep itself never sends the customer anything. Its job is to make sure a person or a fresh turn does.

The logs are graded so the important cases page someone. Replays log at WARNING. A hand-over because a reply was sent but never recorded, or because retries ran out, logs at CRITICAL. A hand-over because the turn was simply too old logs at ERROR.

Rule 4: the sent marker lives outside the transaction

One failure is nastier than the others. The model answers, the provider accepts the message, and then the turn fails before its transaction commits. The customer has the reply, but the database says it’s still owed.

Replaying that turn would message the customer twice. So the moment the provider accepts a reply, the pipeline writes a sent marker to Redis, deliberately outside the transaction. The sweeper checks it. If a reply went out after the turn became owed, the sweep hands the conversation to a person instead of replaying. A human can see what happened and reconcile it, and the customer is never messaged twice.

Rule 5: alarm on age, not on errors

Error-rate alarms miss this whole class of failure. A lost turn doesn’t throw anything you can count.

So in production the sweep publishes one number to CloudWatch: the age of the oldest owed reply. The alarm fires when it stays above ten minutes for two consecutive five-minute periods. That single metric catches everything this post describes: a dead queue, stuck workers, replays that keep failing. It catches them in terms the business cares about, which is how long a customer has been waiting.

The bug that proved the point

After a successful send, the pipeline refreshes the conversation’s rolling summary using a cheap model. One day that model returned an empty structured response, {}. The summary code raised a KeyError, and the exception rolled back the transaction, including the record of a reply the customer had already received.

Without the rules above, that’s a guaranteed duplicate message on the next attempt. With them, the sent marker existed, so the sweep would hand the conversation to a person instead of replaying. The fix itself was small: the post-send summary refresh now catches every exception, not just model errors. The design is what turned a nasty bug into a contained one.

A checklist for your own pipeline

1. Write the obligation in the same transaction as the event. Not in the queue, not in a cache.
2. Settle with compare-and-clear, so newer work survives older turns.
3. Sweep from a different queue than the work you’re sweeping.
4. Make replays idempotent with a token, and exit before any expensive call.
5. Record side effects outside the transaction when they can’t be undone, and check them before replaying.
6. Prefer a person over a guess. An escalation is a feature, not a failure.
7. Alarm on the age of the oldest obligation. It’s the metric your customers actually feel.

Building on WhatsApp or another messaging channel and worried about replies that never arrive? That’s work I take on.