Human-in-the-loop automation: three rules that keep AI from sending the wrong email
Our sales replies are drafted by AI and approved by a person. Three binding rules came out of running it, each learned from a real mistake, plus how to decide what may run without a human.
- Let AI draft, let a person send. Anything that cannot be undone goes through an approval step that a service token cannot bypass.
- Rule 1: automation changes state only from what it observes, never from what it infers.
- Rule 2: record the result where the action happens, in the same request, not in a clean-up job later.
- Rule 3: never act on an item that did not come through your own flow.
"Human in the loop" is easy to say and easy to get subtly wrong. It does not mean a person glances at a dashboard now and then. It means the system is built so that the irreversible step cannot happen without a person, while everything before it runs on its own.
We run this pattern every day. Incoming sales enquiries to a KH Group mailbox are classified, a model drafts a reply from the company's knowledge base, the draft lands in the mailbox, and a person reviews it, asks for changes if needed, and presses Send. Everything sits on n8n plus a small operations board backed by a database. The three rules below became binding after we broke each of them once.
The shape of a safe loop
Before the rules, here is the structure they protect:
- Trigger. A label on an incoming email. A classifier runs four times a day and picks up only labelled messages.
- Draft. The agent loads the knowledge base and writes a reply. It creates a draft in the mailbox. It never sends anything.
- Review. A person opens the case, reads the draft and the customer's message side by side, and either asks for changes (the draft is rewritten in place) or approves it.
- Send. Approval is a button that performs the send and records the result. Sending and stopping a case are allowed only from a logged-in human session. Our own service token cannot do it, so no script or agent can do it either.
The fourth point is the core of the design. If an agent with an API token can call "send", you do not have a human in the loop. You have a human who happens to be around.
Rule 1: change state only from observation, never from inference
We have a clean-up job that keeps the review queue tidy. One morning it removed several live cases. Its logic looked reasonable: the database showed that mail had been sent to each of those customers, so it concluded the cases were answered.
But "mail was sent to this customer" and "this case has been answered" are different facts. The customers had written again, and the new questions had no reply yet. The clean-up job had inferred a state from a field that meant something else, and real enquiries disappeared from the queue.
The rule now: automation may set a status only from something it directly sees. A draft exists without a work-in-progress marker, so it is ready to send. The mail provider confirms a message left, so it is sent. The draft is in the bin, so the case is closed. Anything derived from data becomes a note for the operator, never a status change. The clean-up job still runs four times a day. It just writes notes instead of making decisions.
Test: for every status your automation sets, ask "what exactly did it see?" If the answer contains "so it must be", the automation is guessing, and the guess should be a note.
Rule 2: record the result where the action happens
In one day, three times, an action succeeded and nobody wrote down that it had. A reply went out but the case stayed "ready to send". A draft was approved but the board still showed it as "in review". Each time the plan had been that a background job would notice later and fix the record.
Background jobs make a poor bookkeeping mechanism. They run late, they run on partial data (see Rule 1), and when they fail the board is wrong without anyone knowing. The rule now: the request that performs an action writes its own result, in the same request. The send button sends and records sent. The approve button saves the draft and records ready to send. The clean-up job is a safety net, not the mechanism.
It is the same lesson as in our Academy piece When failure looks like success: a system that reports success separately from doing the work will sooner or later report one without the other.
Rule 3: never act on an item that did not come through your flow
The sales mailbox is shared. Apart from our agent's drafts, it holds drafts from a third-party tool waiting to be picked up, and drafts colleagues write by hand. They all look the same in the mailbox.
Every draft our flow creates carries a session identifier that links it to a recorded case. A draft without one gets its own status, outside the flow. The board hides the send button for it, and the server refuses to send it even if someone calls the API directly. Two layers, on purpose: the interface prevents the mistake, and the server makes it impossible.
This generalises. Whenever automation shares a space with people or other tools, such as a mailbox, a folder or a spreadsheet, mark what is yours and refuse to act on everything else.
Smaller guards that earn their keep
- Rate limits on anything automatic. Our forms gateway sends an automatic "we received your message" confirmation, but at most one per address every ten minutes and thirty per hour, and only after the submission is safely stored. A bug or a bot should not be able to turn your mailbox into a spam cannon.
- Reversible by default. Clean-up moves drafts to the bin, which is recoverable. It never deletes. Stopping a case requires a written reason.
- One error channel with a cooldown. Failures notify a person, but a single broken API produces one alert per half hour, not hundreds.
- Merging before drafting. If the same customer writes three times in a week, the agent receives the messages together in order and writes one reply, not three replies that contradict each other.
When can the human step out?
Not all at once, and not by default. Our rule of thumb follows the autonomy contract we teach in Agents at full autonomy: act freely on what can be undone, gate what cannot. Summarising, classifying, drafting and filing are undoable, so they run unattended. Sending to a customer, deleting, paying and publishing are not, so they wait for a person.
A category of message can graduate to automatic sending when you have a record of many approvals in that category with no meaningful edits, a reliable way to detect that a message belongs to it, and a stop button that still works afterwards. Until then, a person approving a good draft in fifteen seconds is not a bottleneck. It is the reason you can trust the rest of the system.
Build your own loop
Start small: one trigger, one AI step that drafts, one action that waits for approval. The Academy lesson Your first automation builds exactly that in n8n. The autonomous engine shows how far unattended work can go when the line between reversible and irreversible is drawn clearly. If you are weighing up tools, n8n vs Zapier explains why per-run pricing makes review steps affordable.
Want this running in your company?
Flowsmith designs and operates automations like this for small and mid-sized teams — with a person approving anything that leaves the building.
