Exam-style
A regulator asks the team to explain how their RAG assistant arrived at a specific answer. The product manager has promised per-feature attribution scores, the way the company's credit-scoring model reports them. What should the team deliver instead?
Reveal the answer
C. The citations returned by RetrieveAndGenerate showing the source passages that grounded the answer, plus documented model behaviour
Explainability for a foundation model is traceability: which passages grounded this answer, and what the system is documented to do. Bedrock Knowledge Bases return those citations from RetrieveAndGenerate, and a Model Card carries the documented behaviour. SHAP-style attribution assumes a small, stable feature set and does not transfer to a generative model over free text; attention weights are not an account of reasoning; and the raw prompt and completion show what happened without showing what supported it.
Update, 10 August 2026. SageMaker Clarify was moved into maintenance on 30 June 2026 and closed to new customers on 30 July. Its foundation-model evaluation survives the move: it is the open-source fmeval library, which runs anywhere Python does, and Bedrock model evaluation covers the managed version. The reasoning here still holds; what changed is which name you reach for.
Q. Someone asks you to explain a RAG answer. What is the realistic form of FM explainability?
A. Traceability: citations back to the source passages that grounded the answer (Bedrock Knowledge Bases RetrieveAndGenerate returns them), plus documented behaviour. It is not SHAP-style per-feature attribution.
Why? Do not promise feature attribution on an LLM; explainability here is grounding and documentation.