Handling Ambiguous Questions With Clarification

The situation

A retail company runs a customer-service assistant on Amazon Bedrock, backed by a knowledge base for policies and a set of tools that read the signed-in customer’s account: orders, subscriptions, addresses, payment methods. It handles returns, order status, plan changes, and general policy questions. Most days it works well. The complaints that reach the team are all the same shape.

A customer types “where is my order?” and the assistant picks one of the three open orders, usually the wrong one, and reports its status with total confidence. Another asks “can I cancel?” and gets a cheerful walkthrough of cancelling the whole subscription when they meant a single line item. A third asks “how much will it cost to upgrade?” and the model quotes a number for a plan the customer isn’t on, because nothing in the question said which plan they hold and the model filled the gap with a guess.

None of these are hallucinations in the usual sense. The retrieved policy text is accurate, the tools work, the account data is real. The failure is upstream of all that: the question was underspecified, and the assistant answered a more specific question that it invented. It never noticed it was missing something. The team wants it to tell the difference between a question it can answer and a question it only thinks it can answer.

What actually matters

The core failure mode is that a language model treats an ambiguous prompt as a well-formed one. Given “where is my order” and three candidate orders, it does not surface the ambiguity; it silently commits to one reading and generates a fluent answer for it. Fluency is the trap, because the confident tone is identical whether the model resolved the ambiguity correctly or guessed. The first thing worth naming is that the assistant needs an explicit step that judges sufficiency before it answers, rather than treating every input as answerable.

That judgement has cheap, concrete signals feeding it. Ambiguity is often a matter of missing slots: a return needs an order reference and a reason, a plan change needs which plan and which direction. If a required slot is empty and can’t be inferred, the request is underspecified by construction, and you know that before you call the model. Referential vagueness (“my order”, “the subscription”, “that charge”) is another signal, resolvable only when exactly one candidate exists in the account. And when the answer depends on retrieval, low retrieval confidence, thin or scattered matches, or several documents pulling in different directions, is itself evidence that the question may be too broad or aimed at something the corpus doesn’t cover.

Once ambiguity is detected there are three responses, and picking between them is where the design lives. The assistant can ask a clarifying question, which is safest when the missing piece genuinely can’t be recovered and a wrong answer would be costly. It can offer the most likely interpretations and let the user pick, which is faster than an open question when the candidates are few and enumerable. Or it can resolve the gap from context it already holds, the signed-in customer’s account, the entities named earlier in the conversation, without troubling the user at all. Resolving from context is the best outcome when the context makes the answer unambiguous, because it costs the user nothing.

The trade sitting under all of this is over-asking against over-assuming. An assistant that clarifies everything is exhausting and users abandon it; an assistant that assumes everything is confidently wrong and erodes trust faster. The right balance is not fixed, it scales with the cost of being wrong. Reporting the status of the wrong order wastes a sentence and is easily corrected; cancelling the wrong subscription or quoting a binding price is expensive, so those lean towards asking. The blast radius of a mistaken assumption sets how eager the assistant should be to confirm.

The last thing that matters is that resolution must be grounded, not guessed. Filling a missing slot by inventing a plausible value is the original failure in a new place. When the assistant resolves ambiguity, it should do so from real data: an account attribute a tool returned, a document the retrieval step actually pulled, an entity the user actually named earlier. If the gap can’t be closed from grounded context, that is precisely the signal to ask rather than to fabricate.

What we’ll filter on

1. Ambiguity detection, can the design tell an answerable question from an underspecified one before answering?
2. Slot and entity completeness, are the required pieces present, and does exactly one candidate resolve a vague reference?
3. Grounding of the resolution, is a filled gap backed by account data, retrieval, or prior turns rather than a guess?
4. Cost of a wrong answer, does the response mode scale asking versus assuming to the blast radius of a mistake?
5. Conversational friction, does it avoid interrogating the user when context already settles the question?
6. Recoverability, when it does assume, does it state the assumption so a wrong one is easy to correct?

The disambiguation landscape

1. Answer directly. When the question is well-specified or context makes the reading unambiguous, just answer. This is the target state for most turns; the point of everything else is to reach it safely. The failure is answering directly when the question was not actually clear, which is the situation the team is in now.

2. Model-judged sufficiency check. Prompt the model to decide whether it has enough to answer before it answers, returning a structured verdict (answerable, or what’s missing) rather than prose. This turns the implicit “just generate something” into an explicit gate, and because it can name the missing piece, it feeds directly into which clarifying question to ask. It costs an extra reasoning step and is only as good as the prompt, but it catches ambiguity the input-side checks miss.

3. Slot filling. Model the request as a set of required slots and refuse to proceed until they’re filled, prompting for whatever is missing. This is the backbone of conversational designs and is exactly what Amazon Lex does: an intent declares its slots, and the bot elicits any that the utterance didn’t supply before it fulfils the intent. Deterministic and predictable for transactional flows like returns and plan changes; less suited to open-ended questions that don’t decompose into a fixed slot set.

4. Ask a clarifying question. When something required is missing and can’t be recovered, ask for it in plain language: “Which order do you mean, the trainers or the jacket?” Safest response when a wrong answer is costly, and the most natural when the missing piece is a single fact. Over-used, it becomes an interrogation, so it earns its place when context genuinely can’t close the gap.

5. Offer likely interpretations. Rather than an open question, enumerate the candidate readings and let the user choose: “Did you mean cancel the whole subscription, or remove one item from the next box?” Faster than an open prompt when the candidates are few and known, and it doubles as a way to show the user what the assistant can do. It falls apart when the interpretations are many or hard to phrase crisply.

6. Resolve from account context. Use what you already hold about the signed-in user to settle the ambiguity: if the customer has exactly one open order, “where is my order” has one answer and no question is needed. An agent can call a tool to fetch the missing context, an order list, the current plan, the default address, and resolve the reference from real data. The best outcome when it works, because it’s invisible; the risk is resolving from stale or wrong context, so it wants confirmation when the stakes are high.

7. Resolve from conversation history. Carry entities named earlier in the session so later vague references bind to them: if the user discussed order #44821 two turns ago, “when will it arrive” refers to that order. Cheap and natural in multi-turn chat; the danger is a reference that has drifted, where the user has moved on and the old entity no longer applies, so recency and relevance both matter.

8. Confident guess with no signalling. Pick a reading and answer as if it were the only one, saying nothing about the assumption. This is the current behaviour and the anti-pattern: it’s indistinguishable from a correct answer until the user notices, and it offers no thread to pull to correct it. Even when assuming is the right call, stating the assumption (“showing your most recent order”) turns a silent error into an obvious, correctable one.

Side by side

Reading the table against the three complaints: “where is my order” with several open orders wants resolution from account context if one order dominates, and offering the candidates if not; “can I cancel” wants slot filling and a clarifying question because the blast radius is a cancelled subscription; “how much to upgrade” wants the current plan resolved from the account before any price is quoted. None of them wants the confident guess they currently get.

The decision

The three responses are not a ranking; they are a routing decision driven by two questions. Can the gap be closed from grounded context, and how costly is a wrong answer? The flow below is the shape the assistant should follow on every turn: judge sufficiency, try to resolve from context, and only then choose between assuming and asking based on the stakes.

The gates are ordered deliberately. Sufficiency comes first because it is the check the current design skips entirely. Resolution comes before the asking decision because a gap closed from grounded context costs the user nothing, so it should always be tried before troubling them. Only when context can’t close the gap does the cost of a wrong answer decide between offering the readings and asking outright.

The picks in depth

Slot filling is the right backbone for the transactional flows, and it is worth using the platform primitive rather than rebuilding it. In Amazon Lex, an intent declares its slots and the bot elicits any the utterance didn’t fill before fulfilment, so “I want to cancel” with no target sits in an elicit-slot state until the customer names what they’re cancelling. That is exactly the gate the “can I cancel” complaint needs. The slot for what to cancel is required, the utterance leaves it empty, and the design refuses to proceed to a cancellation until it’s filled. Slot filling gives you a deterministic, testable gate for anything that decomposes into required fields, which returns, plan changes, and address updates all do. It fits the structured requests better than the open-ended policy questions, which don’t reduce to a fixed slot set and lean on the model-judged check instead.

Bedrock Agents reach the same gate from the other direction, and the primitive is easy to miss because it looks like an empty action group. Add one with parentActionGroupSignature set to AMAZON.UserInput and leave the schema and executor blank; from then on, when the agent decides it needs to call an action but does not have enough to fill the request, it invokes that group instead of guessing, and the turn comes back as an ASK_USER observation carrying a question for the customer. Without it the agent has no sanctioned way to stop and ask, which is a large part of why an agent that looks well-instructed still invents a plausible order ID. Whatever the customer answers then rides forward in the sessionState on the next InvokeAgent call, as a session attribute that lasts the conversation or a prompt session attribute that lasts a single turn, so the clarification is only asked once.

The model-judged sufficiency check covers what slots can’t. Not every question is a transaction with declared fields; “how does your returns policy work for sale items” is answerable or not depending on whether the corpus covers sale items, and no slot captures that. Here you prompt the model to return a structured verdict, answerable or a named missing piece, before it drafts an answer, and you route on the verdict. Keep the verdict separate from the answer so you can act on it programmatically rather than parsing it out of prose. Retrieval confidence feeds this same gate, and it is a number rather than a feeling: the knowledge base’s Retrieve API returns a relevance score on every result alongside the text, so “thin or scattered” becomes a threshold on the top score and a spread across the rest, both of which you can log, tune against real questions, and test. Below the threshold, the honest move is to narrow the question with the user rather than synthesise a confident answer from weak evidence.

Resolving from context is where an agent earns its keep, and the key is that the resolution is grounded in a tool result, not a guess. Given “where is my order”, the assistant calls the order-lookup tool for the signed-in customer and inspects the result. One open order resolves the reference outright and no question is needed; several means the reference is genuinely ambiguous and the design falls through to offering the candidates (“your trainers or your jacket?”). The tool call is what turns a vague pronoun into a grounded entity, and the count of candidates it returns is what decides between resolving silently and asking. The same pattern fixes the pricing complaint: fetch the customer’s current plan before quoting an upgrade, so the number is computed from the plan they actually hold rather than one the model assumed. Prior turns feed the same mechanism; an order named earlier in the session binds a later “when will it arrive”, as long as the reference is still recent enough to be the thing the user means.

The connective rule across all three is to make the assumption visible whenever you resolve rather than ask. Even a well-grounded resolution can be wrong, a stale default address, an order the customer didn’t mean, so stating it (“showing your most recent order, placed Tuesday”) converts a silent misfire into a one-line correction. This is the cheapest reliability lever in the whole design: it doesn’t reduce the assumption rate, it makes every wrong assumption obvious and recoverable, which is what separates a helpful assistant that occasionally guesses from one that guesses and hides it.

A worked example: where is my order

The customer, signed in, types where is my order? and has three open orders. The current design picks one and reports its status, wrong two times in three.

Under the new flow, the sufficiency step flags the reference as vague: “my order” resolves only if exactly one candidate exists. Before asking anything, the agent calls the order-lookup tool for this customer and reads back three open orders. That count is the routing signal. Three candidates means the reference cannot be resolved silently, so the assistant offers the readings rather than guessing:

You have three orders on the way. Which one do you mean?
  - Trainers, order #44821, out for delivery
  - Jacket, order #44902, in transit
  - Coffee beans, subscription box, ships Friday

If the lookup had returned a single open order, the same flow resolves it silently and answers, stating the assumption so a wrong one is easy to catch: “Your order #44821 (trainers) is out for delivery today.” And if the customer had named an order two turns earlier, the conversation-history binding would have resolved “it” to that order without a tool call at all. One mechanism, the grounded lookup and its candidate count, drives all three outcomes, and none of them is the confident guess the design started with.

What’s worth remembering

1. A model answers a vague question as if it were clear, and its confident tone is identical whether it resolved the ambiguity or guessed; fluency hides the failure.
2. Put an explicit sufficiency step before the answer, so the assistant judges whether it has enough to answer rather than treating every input as answerable.
3. Missing required slots, vague references with several candidates, and low retrieval confidence are all cheap, concrete signals that a question is underspecified.
4. There are three responses to ambiguity, ask a clarifying question, offer the likely interpretations, or resolve from context, and choosing between them is the design.
5. Resolving from grounded context, the signed-in account or entities named earlier, is the best outcome because it costs the user nothing; an agent can call a tool to fetch the missing context.
6. Fill a gap only from real data; inventing a plausible slot value is the original failure in a new place, and an unfillable gap is the signal to ask.
7. Slot filling, as in Amazon Lex intents, is the deterministic backbone for transactional flows; the model-judged check covers open questions that don’t reduce to fixed slots.
8. Balance over-asking against over-assuming by the cost of a wrong answer; a mistaken status wastes a sentence, a mistaken cancellation or price does real damage, so stakes decide how eager to confirm.
9. When you do assume, state the assumption; it turns a silent wrong answer into an obvious, correctable one.
10. Offer enumerated interpretations when the candidates are few and known, and fall back to an open clarifying question when they are many or hard to phrase.