Exam-style
A regulator asks the team to explain how their RAG assistant arrived at a specific answer. The product manager has promised per-feature attribution scores, the way the company's credit-scoring model reports them. What should the team deliver instead?
Reveal the answer
C. The citations returned by RetrieveAndGenerate showing the source passages that grounded the answer, plus documented model behaviour
Explainability for a foundation model is traceability: which passages grounded this answer, and what the system is documented to do. Bedrock Knowledge Bases return those citations from RetrieveAndGenerate, each one carrying the retrieved text and its source location, and a model card records intended use, limits and evaluation results. SHAP-style attribution assumes a small, stable feature set, and it does not transfer to a generative model over free text. Attention weights are not an account of reasoning, and Bedrock does not return them. The logged prompt and completion record what happened without recording what supported it. An AWS AI Service Card sets out a model’s intended uses and limitations in general terms, and AWS publishes those cards for its own services and models rather than for every model on Bedrock, so it cannot account for a specific answer either.
Update, 10 August 2026. AWS announced on 30 June 2026 that SageMaker Clarify is moving to maintenance, and it closed to new customers on 30 July 2026. Existing customers keep access. Clarify computes its feature attribution with the open-source SHAP library, and its foundation-model evaluation is the open-source fmeval library, both of which install with pip and run outside Clarify; Amazon Bedrock evaluations is the managed alternative for foundation models. The reasoning below is unchanged. What changed is the tool you would reach for.
Q. Someone asks you to explain a RAG answer. What is the realistic form of FM explainability?
A. Traceability: citations back to the source passages that grounded the answer (Bedrock Knowledge Bases returns them from RetrieveAndGenerate, with the retrieved text and its location), plus documented behaviour in a model card. SHAP-style per-feature attribution does not apply here.
Why? Do not promise feature attribution on an LLM; explainability here is grounding and documentation.