Exam-style
A regulator asks the team to explain how their RAG assistant arrived at a specific answer. The product manager has promised per-feature attribution scores, the way the company's credit-scoring model reports them. What should the team deliver instead?
Reveal the answer
C. The citations returned by RetrieveAndGenerate showing the source passages that grounded the answer, plus documented model behaviour
Explainability for a foundation model is traceability: which passages grounded this answer, and what the system is documented to do. Bedrock Knowledge Bases return those citations from RetrieveAndGenerate, and a Model Card carries the documented behaviour. SHAP-style attribution assumes a small, stable feature set and does not transfer to a generative model over free text; attention weights are not an account of reasoning; and the raw prompt and completion show what happened without showing what supported it.
Q. Someone asks you to explain a RAG answer. What is the realistic form of FM explainability?
A. Traceability: citations back to the source passages that grounded the answer (Bedrock Knowledge Bases RetrieveAndGenerate returns them), plus documented behaviour. It is not SHAP-style per-feature attribution.
Why? Do not promise feature attribution on an LLM; explainability here is grounding and documentation.