Exam-style
A lender's regulator requires that every declined application reach the applicant with the reasons for that decision, named in terms the applicant can act on. Four candidates are on the table: a logistic regression, a gradient-boosted tree with per-feature attribution, a deep neural network, and a foundation model on Amazon Bedrock. Which choice meets the requirement?
Reveal the answer
C. The logistic regression, or the gradient-boosted tree with its per-feature attribution computed and stored at decision time where the accuracy gain justifies the extra machinery
Two words are doing separate jobs here. Interpretability is understanding a model’s internal mechanics, how it turns an input into an output. Explainability is a post-hoc account of one particular output, produced without opening the model up. A logistic regression gives both, because each feature has a coefficient and one applicant’s contribution is a multiplication anybody can check on paper. A gradient-boosted tree gives the second. Its attribution is machinery bolted on afterwards, and it holds up in an audit three years later only if the attribution values, the feature values and the model version were written down at decision time. A deep neural network and a foundation model give neither. Global feature importance describes the portfolio and never says why this applicant failed, so the second option answers a different requirement from the one the regulator set. Asking the foundation model to explain its own answer is the tempting move and the worst one: a foundation model has no stable input features to attribute, so what comes back is a fluent rationale generated alongside the decision rather than a report of the arithmetic that produced it. The most it can honestly offer is traceability to the sources it retrieved. Having a human write the reasons afterwards fails the same test, because a person reconstructing a plausible story about a score is not describing what the model did. The last option is where the AWS guide’s tradeoffs between model safety and transparency bite. Publishing the coefficients and the cut-off is maximum disclosure, and it hands anyone who wants to be approved the recipe for shaping an application to clear the threshold. Disclose enough for the decision to be accountable, and keep the attack recipe out of the letter.
Q. Every declined application has to name its reasons. Logistic regression, boosted tree with attribution, deep neural network, or a foundation model on Amazon Bedrock?
A. The logistic regression, or the boosted tree with per-feature attribution stored at decision time when the accuracy gain is worth the extra machinery. Both sides of that trade get a number.
Why? Interpretability and explainability are different obligations, and AWS asks you to describe the differences between models that are transparent and explainable and models that are not. Interpretability is reading the internal mechanics: the coefficients, the split points, the score as arithmetic. Explainability is accounting for one output after the fact without opening the model at all. The regression offers both. The boosted tree offers the second, through attribution computed and stored per decision. The deep network and the foundation model offer neither. A foundation model asked to justify its own decline writes something plausible about a decision it cannot inspect. What it can genuinely deliver is traceability to the passages it retrieved, not per-feature attribution. The guide’s phrase for weighing the loss of accuracy against the gain in transparency is measure interpretability and performance, and measure is the operative word: write down the accuracy each option gives up and the explanation each one can produce, then decide. That same weighing runs the other way at the far end. Publishing the whole model satisfies transparency and tells an applicant precisely which number to move. So the tradeoffs between model safety and transparency resolve here at disclosing enough to be accountable, without printing the instructions for gaming it.