All posts

"Human handoff done right: design the escalation path before you need it"

Every AI system has cases it cannot handle, and they all go to a human. Whether that handoff is a designed workflow or a JSON blob in a Slack channel nobody owns decides how the whole system performs.

AI engineering4 min read1 September 2026by Ahmed
"Human handoff done right: design the escalation path before you need it"

When an AI system escalates its first real case, the destination is usually a Slack channel. A JSON blob arrives, formatted for machines, in a channel nobody officially owns. Someone spots it two days later, cannot work out what the system wanted, pings a developer, and the customer's request quietly ages while everyone establishes whose job this is. I have watched versions of this at more than one company, and the root cause is always the same: the escalation path was designed the week after it was first needed.

The underlying mistake is treating escalation as an edge case. It is not an edge case; it is a feature with permanent traffic. Some percentage of inputs will always be beyond the system, malformed, ambiguous, out of scope, or just below the confidence bar, and if you have built your fail-closed machinery properly, that percentage arrives somewhere every single day. The handoff is product surface, and it deserves the same design attention as any other surface.

Start with what the human actually needs to see. Not raw logs, and not the orchestrator's internal state. They need the original input, the system's best attempt, and the specific reason it escalated, which check failed, what the confidence was, what the system would have done if allowed. They need to know what happens if they do nothing, and by when. And the resolution should be one click away: approve, correct, reject. Every minute a reviewer spends reconstructing context is a tax levied on every escalation, forever.

"Human handoff done right: design the escalation path before you need it"

The best handoffs frame a decision rather than dump a case. "Is this invoice total £4,120.50? The document reads 4.120,50 and I am unsure of the separator convention, approve or correct" gets answered in ten seconds. "Extraction failed for document 8841, payload attached" gets deferred to the afternoon, then to tomorrow. You are designing for a human's attention budget, which is the scarcest resource anywhere in the system.

Then the queue itself. Who owns it, a named team with this in their job description, or "whoever notices"? What is the response target, and does it match what the blocked task can tolerate? In what order do items surface, oldest first, highest value first, closest to breach first? And what happens when the queue backs up, because input does not stop arriving just because reviewers are busy? None of these questions is difficult, but every one of them has a bad default answer if nobody chooses deliberately, in advance.

Closing the loop is where most implementations stop short, and it has two halves. The first is mechanical: the human's decision must flow back into the system and complete the task, unblock the workflow, send the corrected value downstream, resume the pipeline. If a reviewer fixes a field in a dashboard and then has to re-key the same fix somewhere else, the handoff is not closed; it is doubled.

The second half is where the compounding value sits. Every escalation is a labelled example, produced by an expert, on exactly the cases your system finds hardest. Captured properly, corrections become eval cases and regression tests, and the system's weakest areas generate their own test material. In docextract (github.com/afareed007/docextract) the human-review gate exists precisely for this: reviewed cases feed the eval suite, so a failure that escaped once gets caught automatically the next time it appears. Thrown away, the same corrections are just labour.

Escalation rate is also one of the best health metrics the system has. A sudden rise means something upstream changed, an input source, a document format, a model version. A slow decline should mean the system is genuinely improving. A drop to zero almost never means perfection; it usually means the escalation logic broke, and quiet wrong answers are flowing through unreviewed. Watch it the way you watch error rates.

Be honest about the human side too, because people adapt to queues in ways that undermine them. A reviewer who approves four hundred items a week stops reading them, approval becomes a reflex, and the gate becomes theatre. Human-factors researchers have called this automation complacency for decades; aviation learned it long before software did. You can push against this: keep volume down by fixing upstream causes rather than staffing the queue harder, mix in occasional known-answer items to measure attention, rotate the duty, and treat a rubber-stamping reviewer as evidence the system escalates too much rather than evidence the human is careless.

Designing this whole loop, the confidence gate, the review queue, and the path back into the system, is the work I do. If your escalation path is currently a Slack channel and good intentions, get in touch via the contact form.

Senior engineering roles — or production AI consulting?

I'm open to senior AI and software engineering roles and to selective consulting on agents, RAG, and LLM automation in production. If this post resonates — as a hiring manager or as someone shipping similar work — get in touch.

Get in touch

© 2026 Ahmed Fareed. All rights reserved.

LOADING