Prevent duplicate messages, reports, and records

Why duplicates happen

Duplicates are not a freak event—they are the normal behavior of reliable systems. A connection times out and the sender retries, a person taps a button twice, a slow run overlaps the next scheduled one, or the agent simply cannot see that an output already exists. Most delivery and scheduling systems promise to run your job at least once, which is another way of saying that repeats will eventually arrive.

That reframes the work. Duplicate protection is not a cleanup you do after a bad week; it is a decision you make before launch, when the cost is one line of thought instead of a batch of apology emails.

A realistic failure: an incoming form takes a few seconds to process, the trigger counts it as failed and redelivers it, and the agent creates the lead twice—same name, same email, two records. Nothing crashed. The system did exactly what it promised.

The trade-off: making repeats truly impossible everywhere is expensive and often unachievable. It is cheaper to accept that a repeat can happen and make sure it lands harmlessly.

Give every intended result one stable identity

Give each intended result a name that stays the same across retries: the task plus its period, or the task plus its source event. “weekly-report-2026-W34” names one report; “lead-form-8472” names one submission. Before the agent creates an output, it checks whether that identity has already completed—and if it has, the run recognizes itself as a repeat instead of starting fresh.

How to judge it: run the same intended result twice and compare the identity it produces. If the two strings match, the guard has something to grab onto. If they differ, you have no identity yet—only two runs that look unrelated.

A realistic failure: baking the exact send time into the name, so a retry two seconds later computes a new identity and slips straight past the check. The clock should not be part of what makes a result unique.

The trade-off: a tighter identity blocks more duplicates but can also block a run you genuinely wanted. “recipient + campaign + week” stops the same person being emailed twice, yet it will also stop a deliberate second email that same week. Pick the grain on purpose.

Choose what a repeat should do

Identity only tells the agent that a repeat has arrived. You still have to decide what the repeat should do. Four rules cover almost everything, and you should be able to name the one for each workflow in a single word:

• Ignore—the first completed result wins and later identical attempts do nothing. Right for notifications and one-time confirmations.
• Update—revise the existing record instead of creating a second one. Right for reports and summaries that should reflect the latest data.
• Queue—let the active run finish before the next one starts. Right for schedules that occasionally overlap.
• Ask—pause for a person when two inputs share an identity but carry conflicting data. Right for records where the difference might matter.

How to judge it: if you cannot say which of the four a workflow uses, it has no policy yet—only a default nobody chose.

A realistic failure: choosing “ignore” for a report, then sending a correction an hour later that the workflow silently drops because the first version already won. “Update” was the rule that fit.

The trade-off: “update” keeps data current but can overwrite a version someone already acted on; “queue” preserves order but adds delay. There is no free rule—match it to what the workflow can tolerate.

Guard external actions hardest

Reading, sorting, and drafting can repeat cheaply—a duplicate draft costs a glance to delete. Messages, payments, publishing, and record creation cannot. They leave your control the instant they happen, so an internal check is not enough: before committing, verify the exact target and search for an earlier completed action.

Payments deserve the strongest guard. Attaching a stable key to the request means that if the same charge is sent twice, the second attempt returns the first result instead of taking the money again. The confirmation can time out, the agent can retry, and the customer is still charged once.

How to judge it: for every external action, ask “if this exact request arrived twice, what stops the second one?” and require a concrete answer, not a hope.

A realistic failure: the payment succeeds but its confirmation times out; the agent, seeing no success, retries; with no key on the request, the customer pays twice and you spend the afternoon issuing a refund and rebuilding trust.

The trade-off: these guards add a lookup and a little latency to every action. Spend that cost only on the things that leave the dashboard, not on the drafts that stay inside it.

Test the double-fire before you trust it

Duplicate handling only runs on the unhappy path, so it stays invisible until you force it. Before you trust a workflow, test it twice on purpose: submit the same safe input two times, and start a second run before the first one finishes. A pass looks like one clear output plus a visible note explaining what happened to the duplicate—suppressed, updated, or queued.

How to judge it: one result and one log line naming the repeat. If you cannot find the note, you cannot prove the guard fired; you may have simply gotten lucky with timing.

A realistic failure: the test “passes” because the second run happened to land after the first had already finished. The two never actually overlapped, so the concurrency rule was never exercised. Real overlap is the only honest test.

The trade-off: staging a true double-fire takes a few extra minutes to set up. It is the cheapest insurance you will buy all week—far cheaper than finding the gap in production.