Exam-style
A retailer's support assistant, built on Amazon Bedrock over the company's own help articles, tells a customer her order cannot be returned. She knows the item is inside the return window, so the answer is wrong. The interface gives her the paragraph and nothing else. What should the team add so a customer in her position can push back?
Reveal the answer
C. Add a control on every answer that lets her mark it wrong and say why, feeding a queue a support person works, and label the assistant as AI, cite the help articles behind the answer, and give her a way to reach a human
Sort the five by who they serve. Guardrail content filters screen for harmful categories such as hate, insults and violence, at a strength of none, low, medium or high. They do not score an answer for accuracy. A model evaluation job measures one model against a prompt dataset. A Model Card records intended use and a risk rating for an assessor. Temperature shapes the distribution the model samples from, so a lower value gives more deterministic wording, not more correct wording. None of the four reaches the customer holding a wrong answer about her order. AWS names two principles of human-centered design for explainable AI, both on her side. User-feedback mechanisms give her a way to disagree in place, a thumbs-down with a comment box, wired to a queue a support person works. AI decision transparency covers what the interface tells her up front: that an AI system produced the answer, which help articles it cites, and how to reach a human who can override it. The answers she marks wrong become the next evaluation dataset.
Q. A customer reads a wrong answer, and the interface gives her a paragraph and nothing else. Stronger guardrail filters, a model evaluation job, a feedback control plus disclosure, a Model Card, or a lower temperature?
A. The feedback control plus disclosure: let her mark the answer wrong into a queue a person works, tell her she is talking to an AI, show the articles it drew on, and name a route to a human. The other four never reach her.
Why? Ask who each control serves. Guardrail content filters screen for harmful categories, a model evaluation job scores a model against a dataset, a Model Card documents intended use, and temperature changes the sampling. All four sit on the builder’s side. AWS names two principles of human-centered design for explainable AI, both on the customer’s. User-feedback mechanisms let her say the answer is wrong where she read it, into a queue a support person works; a control wired to nothing collects complaints and resolves none. AI decision transparency is what she is told without asking: that an AI produced the answer, which help articles it cites, and who can override it. The answers she marks wrong come back as labelled failures, and become the next evaluation dataset.