Designing a Bot-to-Human Escalation Path

The situation

A retailer runs a customer-service assistant on Amazon Bedrock. It answers subscription questions, checks order status through a couple of internal tools, explains the returns policy, and updates delivery preferences. For its first few thousand conversations it did all of that well, and the team was pleased with how rarely it needed a person.

Then the shape of the traffic changed. Customers started asking it to authorise refunds, dispute charges they did not recognise, and cancel contracts mid-term. One asked whether a supplement they had bought was safe to take with their blood-pressure medication. Another wrote three increasingly angry messages after the bot misread an order number, and by the fourth was threatening legal action. The assistant answered all of these in the same even, confident tone it uses for delivery-window changes, because nothing in its design told it that some of them were not its to answer.

Nothing has gone badly wrong yet, which is the dangerous part. The team can see that a refund approved by the bot alone, a wrong answer about a drug interaction, or a legal threat left to escalate itself are all sitting one unlucky conversation away from a real incident. The question underneath every one of them is the same: when should this assistant stop, and how should it hand the conversation to a human without making the customer start over?

What actually matters

The first thing to settle is not a service, it is a list. There is a set of requests this bot must never resolve on its own authority, and naming that set is the whole design. Money leaving the business, a change to a legal or contractual position, anything that reads as medical, legal, or safety advice, and any action the bot has not been explicitly authorised to take all belong on it. Everything on that list needs a route to a human before an answer is committed, not after a customer complains about the one the bot gave.

The second is blast radius. A wrong delivery-window answer costs a follow-up message; a wrong refund costs money and is hard to claw back; a wrong medication answer can hurt someone. The cost of the bot being wrong is not uniform across intents, so the escalation threshold should not be uniform either. Low-stakes intents can tolerate the bot having a go and being corrected later. High-stakes ones should escalate on the first hint of doubt, because the cheap failure is escalating something the bot could have handled, and the expensive failure is answering something it could not.

The third is detectability. An escalation path is only as good as the signals that trigger it, and there are several, each catching a different kind of limit. Low model or intent confidence catches the bot not understanding the request. A guardrail intervention catches the request straying into a blocked or sensitive topic. Intent classification catches a request that is simply out of the bot’s remit. Sentiment and a turn counter catch a customer who is getting nowhere and getting angry. An authorisation check catches an action the bot is not permitted to take even if it understood it perfectly. A design that leans on only one of these signals has blind spots the others would have covered.

The fourth is context preservation. The fastest way to turn a rescued conversation back into a lost customer is to route them to a person who says “how can I help you today?” as though the previous ten minutes never happened. The handoff has to carry the transcript, the identified intent, the account context, and the reason for escalation, so the human picks up mid-thread rather than from zero. This is the difference between a warm transfer and a cold one, and it is mostly a matter of wiring the context through, not a hard technical problem, which is exactly why it gets skipped.

The fifth is the failure default. Under uncertainty, the safe direction is to escalate, not to guess. A bot tuned to answer as much as possible will, by construction, occasionally answer the thing it should have handed off. A bot tuned to hand off when unsure will occasionally escalate something it could have managed, which costs a little human time and nothing else. On anything consequential, failing towards a human is the correct bias, and the design should make that the path of least resistance rather than an exception the bot has to reach for.

There is also a distinction worth drawing early, because it changes which service you reach for. Sometimes the right move is a full handoff, where the human takes over the conversation. Sometimes it is narrower: the bot has worked out what to do and only needs a person to approve the one risky action before it proceeds, then it carries on. Those are different patterns with different tools, and conflating them leads to escalating whole conversations when a single approval step would have done.

What we’ll filter on

1. Trigger coverage: does the design catch low confidence, guardrail intervention, out-of-scope intent, frustration or repeated failure, and unauthorised actions, or only some of these?
2. Stakes-awareness: can the escalation threshold differ by intent, so high-stakes requests escalate sooner than low-stakes ones?
3. Handoff mode: does the situation need a full human takeover, or just human approval of one action before the bot continues?
4. Context transfer: does the human (or reviewer) inherit the transcript, intent, and escalation reason, so the customer does not repeat themselves?
5. Fail-safe default: when signals are ambiguous, does the path escalate rather than let the bot guess?
6. Authority enforcement: is a consequential action blocked from executing until a permitted party has approved it?

The escalation landscape

Confidence signals. Large language models on Bedrock do not emit a calibrated “I am 80% sure” score you can route on directly, so confidence for routing usually comes from a classifier in front of or alongside the model. Amazon Lex returns an NLU confidence score on each interpreted intent and falls back to AMAZON.FallbackIntent when nothing clears the threshold, which gives you a clean, numeric trigger. For a Bedrock-native flow you can add a lightweight intent-classification step and treat a low score, or a “none of these” result, as the escalate signal. The point is that confidence-based routing needs a source of confidence, and that source is typically the classifier, not the generative model’s prose.

Guardrail stop reason. Amazon Bedrock Guardrails let you block denied topicsDenied topicsSubjects you describe in plain language that a Bedrock Guardrail refuses to discuss, whichever way a user phrases the request. , filter harmful content, redact or block sensitive information, and check for grounding. When a guardrail intervenes during a Converse or ConverseStream call, the response carries a stopReason of guardrail_intervened, and the trace tells you which policy fired. That is a first-class escalation trigger: a guardrail stopping the model on a medical, legal, or self-harm topic is precisely the moment the conversation should go to a human rather than to a canned refusal that leaves the customer stuck. Treat guardrail_intervened as “route this”, not just “suppress this”.

Intent classification for scope. Separately from confidence, the classifier tells you whether the request is even in the bot’s remit. Order status is in scope; a contractual dispute is not. A request that classifies to a known out-of-scope intent, or to a high-stakes one like “refund” or “legal complaint”, can be routed straight to a human regardless of how confident the classifier is, because the issue is not comprehension, it is authority. This is where the “must never answer alone” list becomes code: those intents short-circuit to escalation by policy.

Sentiment and a turn counter. A customer can be understood perfectly and still be failing. Amazon Comprehend’s DetectSentiment returns positive, negative, neutral, or mixed, and Amazon Lex can surface sentiment on each turn through its Comprehend integration. A run of negative sentiment, or a simple counter that trips after the bot has failed to resolve an intent two or three times, catches frustration and looping before the customer gives up. These are cheap to add and catch a failure mode none of the other signals see: the slow-motion bad experience.

Handoff to a contact centre. When a full takeover is right, Amazon Connect is the standard destination. A Lex bot can be the automated first tier of a Connect contact flow and escalate to a human agent when a trigger fires, and because Connect carries contact attributes through the flow, you can pass the transcript, the resolved intent, the account identifier, and the escalation reason to the agent’s screen. The customer moves from bot to person inside one session, and the agent starts warm. For text-only products a ticket in a system like a case or a queue is the lighter-weight version of the same idea, as long as the same context travels with it.

Human review of one action. When the bot only needs a person to approve a single risky step, a full handoff is overkill. Amazon Augmented AI (A2I) is built for exactly this: it defines a human review workflow, and through StartHumanLoop you send a specific decision to a human worker (a private team, for instance) to approve, reject, or correct, then feed the result back into the flow. That fits “the bot has decided to issue a £200 refund, hold it for a human to approve” far better than routing the whole conversation away. The bot keeps the thread; only the consequential action waits on sign-off.

Confirmation before an action fires. Amazon Bedrock Agents give you a narrower guard at the action layer. An action groupAction groupThe bundle of API operations a Bedrock agent is allowed to call, described by a schema so the model knows what each one does. ’s functions can be marked to require user confirmation, so the agent must get an explicit yes before it invokes the tool that moves money or changes a record, and return of control hands the actual execution back to your application where your own authorisation checks live. This is the difference between the model deciding to do something and the something actually happening: the confirmation and the return-of-control boundary are where a human or a permissions check sits between intent and effect.

Side by side

Mechanism	Trigger it serves	Catches	Full handoff	Action approval	Carries context
Lex NLU confidence + fallback	Low confidence	Bot not understanding	✗	✗	via Connect
Bedrock Guardrails stopReason	Guardrail intervention	Sensitive or blocked topic	✗	✗	trace only
Intent classification	Out-of-scope / high-stakes	Request outside remit or authority	✗	✗	intent label
Comprehend sentiment + turn counter	Frustration / repeat failure	Customer getting nowhere	✗	✗	signal only
Amazon Connect	Full takeover needed	Everything above, escalated	✓	✗	✓ (contact attributes)
Ticket / case queue	Async takeover	Non-urgent handoff	✓	✗	✓ (if wired)
Amazon A2I	Approve one risky action	Consequential single step	✗	✓	✓ (review payload)
Bedrock Agents confirmation + return of control	Action beyond authority	Money or record change	✗	✓	✓ (action input)

No single row is the answer. The confidence, guardrail, intent, and sentiment rows are detectors; the Connect, ticket, A2I, and Agents rows are routes. A working design pairs detectors with routes: the detectors decide that the bot should stop, and the mode of the request decides whether it stops into a full handoff or a single approval.

The picks in depth

Start with the list, not the plumbing. Write down the intents this assistant must never resolve alone: issue or approve a refund, cancel or alter a contract, and anything that reads as medical, legal, or safety guidance. Those get routed by policy the moment the classifier recognises them, with no confidence threshold involved, because the reason to escalate is authority, not comprehension. Encoding that list is the single highest-value thing in the design, and it is the part that has nothing to do with which AWS service you pick. Everything after it is choosing detectors and routes to serve that list.

For the detectors, layer them rather than choosing one. Lex NLU confidence and the fallback intent catch the bot not understanding; treat a low score or a fallback as “hand off” for any high-stakes flow and “reprompt once, then hand off” for low-stakes ones. Bedrock Guardrails give you the sharpest trigger for sensitive topics: configure denied topics and content filters for medical, legal, and self-harm territory, and route on a guardrail_intervened stop reason instead of returning a bare refusal, because a customer who hit a guardrail still has a real problem that a person should pick up. Add Comprehend sentiment and a turn counter so a frustrated or looping customer escalates before they leave. The layering matters because each detector is blind to what the others catch, and the cheap failure mode of the whole system is a limit that no signal was watching for.

For the routes, split full handoff from single-action approval and use the right tool for each. When the customer needs a person to own the conversation, Amazon Connect is the destination, and the design work is passing contact attributes so the agent inherits the transcript, the resolved intent, the account, and the escalation reason. The customer should never re-explain themselves; a cold “how can I help?” after a ten-minute bot conversation is a self-inflicted wound. When the bot has already worked out the right action and only a risky step needs sign-off, keep the bot in the thread and gate the step: Amazon A2I sends that one decision to a human reviewer to approve or correct, and Bedrock Agents can require confirmation and use return of control so the money-moving tool runs in your application behind your own authorisation checks rather than firing on the model’s say-so. Approving one action is cheaper and faster than escalating a whole conversation, and it keeps the assistant useful right up to the boundary of its authority.

Tie it together with a fail-safe default. Where the signals are ambiguous, the flow should fall towards a human, not towards an answer. That bias costs some human time on conversations the bot could have handled, and it buys you the guarantee that the expensive failure, the bot confidently resolving something it had no business resolving, has no easy path to happen. On a consequential request, escalating unnecessarily is a rounding error; answering wrongly is an incident.

A worked example: the refund that should wait for a person

A customer opens with: “I was charged twice for my March box and I want £58 refunded to my card today.” The classifier resolves this to a refund intent with high confidence. High confidence is not the point here; refund is on the must-not-resolve-alone list, so the intent alone decides the route.

Without an escalation path, the assistant does what it was built to do: it calls the refund tool and tells the customer the money is on its way. If it misread the amount, or the charge was legitimate, or the account is flagged for abuse, that money is gone and the correction is a support case of its own.

With the path in place, the design branches on mode rather than escalating the whole conversation. The bot has understood the request and even gathered the evidence (two charges on the March order), so it does not need a human to take over the chat; it needs a human to approve one action. The refund step is gated: an Amazon A2I human loop presents the proposed refund, the amount, and the two matching charges to a reviewer, and the tool that actually moves the money is a Bedrock Agents action marked to require confirmation, executed through return of control in the application where the authorisation check lives. The reviewer approves, the refund fires, and the bot tells the customer it is done, all inside the same conversation. The customer waited a minute, not a day, and no money moved on the model’s word alone.

Now change one detail: the customer’s third message is “this is fraud and I have already spoken to my solicitor.” Sentiment turns sharply negative and the classifier now sees a legal-complaint intent, which is a full-handoff item, not an approve-one-action item. The flow routes to Amazon Connect, and the agent’s screen opens with the whole transcript, the identified intents, the account, and the reason for escalation already populated. The customer does not repeat a word of it. Two different requests, two different modes, one design that told them apart by asking whether the bot needed approval for a step or needed to get out of the way entirely.

What’s worth remembering

1. The design starts with a list, not a service: name the requests the bot must never resolve on its own authority (money, contracts, medical, legal, safety) and route those by policy.
2. Use several detectors, not one. Confidence, guardrail interventions, intent scope, sentiment, and a turn counter each catch a different limit, and any one alone leaves blind spots.
3. Confidence for routing comes from a classifier such as Lex NLU scores, not from the generative model, which does not emit a calibrated confidence you can route on.
4. Treat a Bedrock Guardrails guardrail_intervened stop reason as “route this to a human”, not just “show a refusal”; the customer who tripped it still has a real problem.
5. Separate a full handoff from approving one action. Amazon Connect (or a ticket) is for a human takeover; Amazon A2I and Bedrock Agents confirmation are for signing off a single risky step while the bot keeps the thread.
6. Carry context through the handoff: transcript, resolved intent, account, and escalation reason, so the customer never repeats themselves and the human starts warm.
7. Make consequential actions require confirmation or approval before they execute; deciding to move money and actually moving it should have a person or a permission check between them.
8. Set the failure default to escalate. Under ambiguity, handing off costs a little human time; answering wrongly on a high-stakes request costs an incident.
9. Let the escalation threshold vary by stakes: low-stakes intents can have a go and be corrected; high-stakes ones escalate on the first hint of doubt.
10. A bot that never hands off is not more capable, it is one unlucky conversation away from confidently answering the thing it should have escalated.