The situation
A consumer lender has about two hundred thousand active accounts across personal loans and credit cards. Two features have landed on the same quarter’s backlog, and the engineering team has proposed Amazon Bedrock for both of them, on the strength of a demo that impressed everyone in the room.
The first is credit decisioning. The lender has eleven years of applications behind it, roughly 1.4 million of them, each stored with the decision that was made and two years of repayment behaviour afterwards. Every applicant who is refused has to be told the principal reasons for the refusal, in writing, in terms that name the things about their application that drove it. The regulator can ask the lender to reconstruct the reasoning behind any single decision for seven years after it was made.
The second is email triage. Around four hundred customer emails arrive each day into one shared inbox, and a person reads each one and forwards it to one of eleven teams. Nobody has ever recorded which team an email ended up with, so there is no history of correct answers to learn from. The current reader is leaving in three weeks, and the feature has a fortnight.
Same organisation, same quarter, same team. The two features want opposite answers.
What actually matters
Start with explainability, because on the credit side it decides the outcome before anything else gets a vote. A traditional supervised model over tabular features, say a gradient-boosted tree over forty columns of income, repayment history and account age, can be interrogated after the fact. Amazon SageMaker Clarify computes SHAP values for a prediction, which divide that one prediction up across the inputs that produced it and say how much each one moved the answer and in which direction. That gives the lender a defensible sentence: this application was refused because the repayment-to-income ratio and three missed payments in the past year pushed it below the threshold, and here is how much each contributed.
A foundation model can be asked to explain itself, and it will produce fluent text that reads like a reason. That text is generated in the same way the answer was, so it describes the answer rather than measuring what caused it. Where the model is grounded in retrieved documents you get traceability to the sources it drew on, which is a real and useful thing, and it is not the same as per-feature attribution. When a regulator wants to know what drove one decision, the difference between those two stops being academic. That is where regulatory concerns bite. They turn explainability from a preference into a filter that removes options.
Then labelled training data, which is what decides the triage side. Traditional ML models learn a mapping from examples somebody already labelled: a few thousand rows at minimum, and considerably more when the outcome you care about is rare. The lender has 1.4 million labelled credit decisions and zero labelled email routings. Foundation models (FMs) arrive pre-trained on a general corpus and can be told what the eleven teams are in the prompt, with three or four worked examples, and start classifying the same afternoon. Where the labels exist, the traditional route uses an asset the lender already owns. Where they do not, collecting them is a project of its own, and the two-week deadline is a real constraint rather than a preference.
Determinism and reproducibility pull the same way. A trained model is a fixed artefact sitting in a registry. The same inputs against the same version give the same output every time, and reconstructing a decision from 2024 means loading the 2024 artefact and running it again. A hosted foundation model is versioned by its provider, sampling settings can be turned down but not off, and models get deprecated on the provider’s schedule rather than yours. Reproducing a two-year-old answer becomes a question of whether that model version is still there.
Cost and latency separate at volume, not in the demo. A small tabular model on a real-time endpoint answers in single-digit milliseconds and costs a fraction of a cent per prediction, with the bulk of the spend sitting in the endpoint you keep running. A foundation model is priced per token in and per token out, and answers in hundreds of milliseconds to a few seconds. Four hundred emails a day makes that cost round to almost nothing. An application funnel that scores every applicant, re-scores on every change and runs overnight batch reviews does not.
Last, operational constraints, which usually decide it when the first two do not. Owning a traditional model means owning labelling, a feature pipeline, a training job, an evaluation harness, retraining when the inputs drift, and an endpoint with a bill attached. Owning a foundation model feature means owning prompts, an evaluation set to catch regressions, guardrails, and the work of moving when the provider retires a version. Neither is free, and the second is not automatically lighter; it is a different set of things to be responsible for, and it lands on a different sort of person.
What we’ll filter on
- Explanation obligation: does someone have a legal right to be told the principal reasons for one specific decision?
- Labelled history: does the organisation already hold thousands of examples with the correct answer recorded?
- Output shape: is the answer a label or a number from a fixed set, or open-ended language?
- Volume economics: at real traffic, do per-call cost and latency budgets rule anything out?
- Time and ownership: how quickly must it ship, and who runs it in six months?
The landscape
Three routes are genuinely on the table for a business problem like either of these, and it is worth being precise about what each one asks of you before comparing them.
A traditional supervised model on Amazon SageMaker AI
You bring labelled historical data, choose an algorithm suited to the shape of the answer, train, evaluate against held-out data, and deploy the resulting artefact to an endpoint. For tabular problems this is usually a gradient-boosted tree or a logistic regression, both of which are small, quick and well understood by the people who audit them. SageMaker Clarify sits alongside it for two jobs: measuring bias in the training data and in the trained model, and producing SHAP attributions for individual predictions. The model registry holds versions, so a decision made in 2024 can be re-run against the artefact that made it.
The cost of this route is everything upstream of the training job. Without labels there is nothing to train on, and manufacturing labels means paying people to read historical cases and record the answer.
A purpose-built AWS AI service
Amazon Comprehend, Amazon Textract, Amazon Rekognition, Amazon Transcribe and Amazon Personalize each solve one well-defined problem behind an API, with no model to train for the common cases. Where an ask lands squarely on one of them the choice is usually easy, and this pairing is worked through in more detail in the case for reaching past a foundation model. The wrinkle for the triage feature is that routing to eleven bespoke internal teams is not a general problem. Comprehend would need custom classification, and custom classification wants labelled examples. That puts it back in the same queue as the traditional route.
A foundation model on Amazon Bedrock
A large pre-trained model, called through an API, told what to do in a prompt. No training data, no training job, no endpoint to size. You describe the eleven teams, give a handful of examples, and get a classification back in the same working day. Four things come with that speed: per-token cost at volume, latency in the hundreds of milliseconds, answers that vary between runs, and an explanation you cannot audit feature by feature. Amazon SageMaker JumpStart offers a related shape, where a pre-trained model is deployed to your own endpoint and optionally fine-tuned. That trades the per-token bill for a running endpoint, and puts the model version under your control.
Evaluation
Side by side
| Option | Per-decision explanation | Needs labelled history | Handles open-ended language | Cost and latency at high volume | Ships in a fortnight |
|---|---|---|---|---|---|
| Traditional supervised model on SageMaker AI | ✓ per-feature attribution via Clarify | ✗ requires thousands of examples | ✗ tabular and fixed labels | ✓ milliseconds, fractions of a cent | ✗ if the labels do not exist |
| Purpose-built AWS AI service | ✗ confidence scores, not attributions | ✗ for custom classification | ✓ within the task it was built for | ✓ per-request pricing, low latency | ✓ only if the task matches the service |
| Foundation model on Amazon Bedrock | ✗ generated rationale, source traceability at best | ✓ needs none to start | ✓ its strongest ground | ✗ per-token cost, hundreds of milliseconds | ✓ same-week prototype |
Two columns do the separating. The explanation column removes the foundation model from anything a regulator will ask about. The labelled-history column removes the traditional model from anything nobody has ever written the answers down for. When both columns say no to the same option, the decision is made; when they disagree, the remaining three break the tie.
Where each feature lands
The solution
Credit decisioning goes to a traditional supervised model on SageMaker AI, and the explanation obligation is what settles it rather than accuracy. The lender has to produce, on demand, the principal reasons for one refusal, seven years after the fact. SHAP attributions through SageMaker Clarify produce exactly that, in the same units the adverse-action notice is written in, and the model artefact is a version in a registry that can be loaded again. A foundation model could very likely predict default well. It could not, at the end of a hearing, tell anyone what drove one refusal in a way that survives being challenged.
Two operational details ride along. The training data itself needs bias measurement before anyone celebrates the accuracy number, which is the other half of what Clarify is for, and is treated as its own subject in the SageMaker suite’s division of labour. And the retention plan has to cover the model artefact, the training data snapshot and the feature pipeline, not just the decisions, because reproducing a decision needs all three.
Email triage goes to a foundation model on Amazon Bedrock, and the absence of labels settles that one. There is nothing to train on, gathering a training set would take longer than the deadline, and the cost of a wrong answer is that somebody in the wrong team forwards the email on. Four hundred emails a day at a few hundred tokens each is a rounding error on the monthly bill. Route the low-confidence cases to a human rather than guessing, and log every routing decision along with the correction when a team bounces one back.
That logging is what makes the third row of the diagram possible. After six months the lender owns something it did not have on day one: tens of thousands of emails with the correct team attached, produced as a by-product of running the feature. A small classifier trained on those labels answers in milliseconds at a fraction of the per-call cost, and the foundation model stays in place for the cases the classifier is unsure about. Use the model’s own output as training labels with care, because a classifier trained on them inherits whatever the model got wrong. The human corrections in the log are the part worth trusting most. Have someone check a sample of the rest before it becomes ground truth.
The general shape holds beyond this lender. Where a decision must be explained, is made from structured data, and has been made thousands of times before with the answer recorded, traditional ML models are the defensible choice. Where the input is open-ended language, no labels exist, and a wrong answer is recoverable, foundation models get you to production in days. Neither is the modern option and the old one; they answer different questions about what you already own and what you have to prove. The related judgement of whether to reach for generative AI at all is worked through in the harder version of this call, and the argument for trying the simplest model first is made in the case for the unglamorous baseline.
Worked example
Two more asks arrive at the same lender a month later. Both go through the same gates.
Blocking fraudulent card transactions
Nine years of transactions, every one of them flagged or not flagged by the fraud team afterwards, so labels exist in the millions. The answer is a score between zero and one. The latency budget is forty milliseconds, because the decision happens while the card is at the terminal. Customers whose transactions are blocked complain, and the complaints team needs to say why.
Every gate points the same way. Labels exist, the output is a number, the budget rules out anything token-priced, and the explanation obligation rules out anything without attribution. This is a traditional supervised model, and the interesting work is in the imbalanced-classes problem rather than in the choice of route.
Answering agent questions from the policy handbook
Four hundred pages of product terms, updated quarterly. Agents ask questions in their own words during calls. Nobody has ever written down which passage answers which question, so there are no labels. The answer is a paragraph of English, not a label. A wrong answer is caught by the agent reading it before they say it aloud.
No labels, open-ended output, a human between the model and the customer, and no per-decision explanation obligation because the agent owns what they say. This is a foundation model over the handbook, and the design work goes into retrieval and into showing the agent which passage the answer came from.
What’s worth remembering
- A per-decision explanation obligation is a hard filter: traditional ML models give per-feature attribution through SageMaker Clarify and SHAP values, while a foundation model gives a generated rationale and, when grounded, traceability to its sources.
- Labelled history is the other hard filter: a traditional model needs thousands of recorded correct answers, and foundation models (FMs) need none to start.
- Structured inputs with a label or a number as the answer point at a traditional model; open-ended language in or out points at a foundation model.
- At high volume the economics separate, because a small trained model costs a fraction of a cent in single-digit milliseconds while a foundation model is priced per token and answers in hundreds of milliseconds.
- Operational constraints decide the ties: one route means owning labelling, training and retraining, the other means owning prompts, evaluation and somebody else’s version schedule.
- The two routes combine, because a foundation model shipped now produces the labelled data that trains a smaller, cheaper model later, provided the human corrections are kept and a sample of its own labels is checked.