Best for

The first ten minutes after an agent run fails mid-action

Time to try

10-minute incident runbook

Start here

Pause the schedule before you touch retry

The AI agent incident response checklist, in order

When a run fails mid-action, the useful skill is not diagnosing the cause—it is not making things worse while you figure it out. Work these five moves top to bottom, and do not skip ahead because a step looks unnecessary:

  1. Contain—stop the workflow so it cannot fire again on its own.
  2. Assess—find out what, if anything, already left the dashboard.
  3. Reconcile—match what the run intended against what actually happened, then retry only the gap.
  4. Communicate—tell anyone a side effect actually reached, and no one it did not.
  5. Prevent—add the one guardrail that would have caught this exact failure.

Step one is containment, and it comes first for a reason. The agent-failure decision tree answers a single question—did anything leave the dashboard—and tells you whether to stop; this checklist is what you run the moment the answer is “stop” or “not sure.” What to do: pause the schedule and cancel any pending automatic retry, so the failed run is frozen exactly where it stopped. Why it matters: a workflow left running can re-trigger itself while you are still reading the error, repeating the very action you are trying to investigate. How to judge it: containment holds when the run has no way to fire again without you—no live schedule, no queued retry, no trigger waiting behind it. Failure case: an owner reads the error, opens a new tab to investigate, and forgets the hourly schedule is still on; sixty minutes later it runs the half-finished job again. The trade-off: pausing a legitimate workflow delays a real result, which is mildly annoying—far cheaper than a duplicated action you now have to unwind by hand.

Assess what actually left the dashboard

Before you plan any fix, take inventory of consequences. Go channel by channel and ask what escaped your control between the run starting and the error appearing.

What to do: open the activity log and the outside systems the workflow touches, and mark each external channel—messages sent, payments moved, posts published, records deleted, access changed—as “yes, it happened,” “no, it did not,” or “not sure.” Treat every “not sure” as a “yes” until a record proves otherwise. Why it matters: your entire response hinges on this inventory. A run that failed while reading or drafting changed nothing outside the dashboard and is safe; a run that failed after sending or paying has consequences that a retry will multiply. How to judge it: you are done assessing when you can name the state of every external channel from a source of truth—the sent folder, the ledger, the published feed—not from the agent’s own summary of itself. Failure case: a payment run times out on the confirmation screen, the owner assumes nothing went through because it said “failed,” and only discovers the completed charges after a customer asks about them. The timeout was a slow reply, not a failed action. The trade-off: checking each channel by hand takes a few minutes you would rather spend fixing the problem, but guessing at the state of an external system is how a ten-minute incident becomes an afternoon of apologies.

Use case: Rafael’s Sunday lesson email breaks mid-send

Rafael runs a small online course studio by himself. Every Sunday evening his agent emails active students a short “this week’s lesson is ready” note. One week the run turns red partway through and the dashboard says only that the task did not finish. His thumb goes straight to retry.

He had three honest options. He could rerun the whole batch and risk sending every student a second copy; he could abandon the run and leave real students un-notified; or he could work the checklist. He worked the checklist. First he contained it—he paused the Sunday schedule so it could not fire again while he looked. Then he assessed: opening the sent record, he found that 140 of the 180 intended emails had actually gone out before the failure, and—because containment had stopped any retry—not one had gone twice.

Reconciling was then simple. He compared his intended list of 180 against the 140 confirmed sent, isolated the 40 that never left, and started a fresh run for only those 40. The communicate step turned out to be a deliberate no-op: no student had received a duplicate or a wrong message, so there was no one to notify, and blasting an apology to 180 people would have invented a problem. Finally he closed the gap—the lesson task now checks the sent record first and skips anyone already emailed this cycle, with a cap so a stuck run cannot retry itself into a storm.

The lesson: the word “failed” described the report, not the work. Running the five moves in order turned a reflex that would have sent 140 duplicate emails into a clean 40-email finish—and the new skip rule means the next mid-send failure fixes itself.

Reflex retry vs. the reconciled retry

Illustrative figures from Rafael’s worked example—an example of reconciling before retrying to prevent duplicates, not a customer result.

180 sendsReflex retry

Rerunning the whole batch resends everyone—140 of them a second time.

40 sendsChecklist retry

Reconciling first limits the rerun to the confirmed remainder.

140Duplicates prevented

The gap between a blind rerun and a reconciled one.

Reconcile, then safely retry a failed AI agent workflow

Once you know what happened, the recovery is almost mechanical: rebuild the gap between intent and reality, and fill only the gap. To safely retry a failed AI agent workflow, you never rerun the whole batch—you rerun the part that provably did not finish.

What to do: lay the run’s intended list beside the confirmed-completed list, and start a fresh run for only the items that are missing. Why it matters: reconciling first is how you prevent duplicate actions after an AI agent failure—a blind rerun repeats every action that already succeeded, while a reconciled rerun touches only the ones that did not. A retry is only free when repeating the action changes nothing; sending an email or moving money is not that kind of action. How to judge it: every intended item is marked done or not-done against an outside record before you retry anything, and the retry set contains not a single item you already confirmed complete. Failure case: the run reports “failed,” the owner reruns the entire batch to be safe, and the eighteen messages that had already sent go out a second time. A stable identity per item makes this recognition automatic—see preventing duplicate runs—so a repeat lands harmlessly instead of doubling. The trade-off: reconciling item by item is slower than one confident click, but that one confident click is exactly what creates the mess you are trying to avoid.

Communicate before someone else notices

If the assessment found a side effect that reached a real person—a duplicate message, a wrong recipient, a missed send someone was expecting—a short, honest note costs far less trust than silence does. This step is conditional: run it only when a consequence actually escaped.

What to do: for each affected person, send a plain message that states the fact, what you are doing about it, and what they need to do—usually nothing. Why it matters: people forgive a mistake they hear about from you first; the same mistake discovered on their own reads as carelessness or evasion. Clear communication is treated as a core part of handling any incident, not an optional courtesy. How to judge it: a good message can be read in ten seconds and answers “what happened, what now, what do I need to do” without the reader having to reply for clarification. Failure case: an owner mass-emails every customer to apologize for an issue that touched only three of them, turning a quiet fix into a public event and worrying hundreds of people who were never affected. The trade-off: notifying draws attention, so aim it precisely—reach the people a side effect actually touched, and say nothing to the ones it did not.

Close the gap so the same incident cannot recur

Recovery ends this incident; it does nothing about the next one. The last move converts the failure you just handled into a rule that handles its whole category quietly, so you are not doing triage again on a busier morning.

What to do: add the single guardrail that would have caught this exact failure—a required-input check before the task starts, an approval gate before the risky step, a cap on how many times a run may retry itself, or a stable identity that makes a repeat harmless. Why it matters: the same handful of failure types recur, and a guardrail you write once turns each future instance into a plain pause or message instead of a red error at 7 a.m. Treating an incident as an input to your standing rules, rather than a one-time cleanup, is the habit that compounds. How to judge it: re-trigger the original condition with a harmless test and confirm the failure now surfaces as a clear pause or plain-language note, not the same crash or duplicate. Failure case: an owner fixes the missing input by hand, adds no check, and hits the identical blank-field failure a month later. The trade-off: every guardrail adds a little friction, so place them where a failure was costly or confusing—not on every step—and test each new rule once before you trust it.

Try this next

  1. Contain first: pause the schedule and cancel any pending auto-retry so the failed run is frozen where it stopped.
  2. Assess every external channel—sent, paid, published, deleted, access changed—from an outside record, and treat “not sure” as “yes.”
  3. Reconcile the intended list against the confirmed-completed list, then rerun only the gap—never the whole batch.
  4. Notify only the people a side effect actually reached, then add the one guardrail that would have caught this exact failure.

Sources and further reading

These primary references support the article’s approach to containing a failed run, assessing side effects, retrying only the safe remainder, communicating clearly, and closing the gap with a standing guardrail.

Ready to put one useful workflow to work?

Start with one clear job, a result you can review, and boundaries you understand.

See launch pricing