The situation
A consumer lender has about two hundred thousand active accounts across personal loans and credit cards. Two features have landed on the same quarter’s backlog, and the engineering team has proposed Amazon Bedrock for both of them, on the strength of a demo that impressed everyone in the room.
The first is credit decisioning. The lender has eleven years of applications behind it, roughly 1.4 million of them, each stored with the decision that was made and two years of repayment behaviour afterwards. Every applicant who is refused has to be told the principal reasons for the refusal, in writing, in terms that name the things about their application that drove it. The regulator can ask the lender to reconstruct the reasoning behind any single decision for seven years after it was made.
The second is email triage. Around four hundred customer emails arrive each day into one shared inbox, and a person reads each one and forwards it to one of eleven teams. Nobody has ever recorded which team an email ended up with, so there is no labelled history to learn from. The current reader is leaving in three weeks, and the feature has a fortnight.
Same organisation, same quarter, same team. The two features point in opposite directions.
What actually matters
Start with explainability, because on the credit side it settles the outcome before accuracy is even measured. A traditional supervised model over tabular features, say a gradient-boosted tree over forty columns of income, repayment history and account age, can be taken apart afterwards. SHAP values divide one prediction across the inputs that produced it, assigning each feature an importance value for that particular prediction, with a direction. That gives the lender a defensible sentence: this application was refused because the repayment-to-income ratio and three missed payments in the past year pushed it below the threshold, and here is how much each contributed. Amazon SageMaker Clarify is the managed wrapper around this, and it is closed to new customers, so a team starting today runs the SHAP library directly in a SageMaker AI pipeline and logs the results to SageMaker AI MLflow for lineage.
A foundation model prompted for an explanation returns fluent text that reads like a reason. That text is generated the same way the answer was, so it describes the answer rather than measuring what caused it. Grounding the model in retrieved documents gives traceability to the sources, which is useful and is not per-feature attribution. When a regulator asks what drove one refusal, that difference turns explainability from a preference into a filter that removes options.
Labelled training data separates the two features. Traditional ML models learn a mapping from examples somebody already labelled: a few thousand rows at minimum, and considerably more when the outcome you care about is rare. The lender has 1.4 million labelled credit decisions and zero labelled email routings. Foundation models (FMs) arrive pre-trained on a general corpus, the eleven teams can be described in the prompt with three or four worked examples, and classification starts the same afternoon. Where the labels exist, the traditional route uses an asset the lender already owns. Where they do not, collecting them is a project of its own, and the two-week deadline is a real constraint rather than a preference.
Determinism and reproducibility pull the same way. A trained model is a fixed artefact sitting in a registry, which keeps versions and lineage for this reason. The same inputs against the same version give the same output every time, and reconstructing a decision from 2024 means loading the 2024 artefact and running it again. A hosted foundation model is not that. Lowering the temperature steepens the token distribution and makes the output more deterministic, but sampling does not switch off. Bedrock also retires models on its own lifecycle rather than the provider’s: each model card carries an earliest end-of-life date and a notice period, usually six months and sometimes forty-five days, after which the model is removed from every Region and requests to it fail. Reproducing a two-year-old answer becomes a question of whether that version still exists.
Cost and latency separate at volume, not in the demo. A small tabular model on a real-time endpoint answers in milliseconds, and the bulk of the spend sits in the endpoint you keep running rather than in the per-prediction cost. A foundation model is charged per input token and per output token, and generating an answer token by token takes far longer than one pass through a small tree. Four hundred emails a day makes that cost round to almost nothing. An application funnel that scores every applicant, re-scores on every change and runs overnight batch reviews does not.
Last, operational constraints, which usually decide it when the first two do not. Owning a traditional model means owning labelling, a feature pipeline, a training job, an evaluation harness, retraining when the inputs drift, and an endpoint with a bill attached. Owning a foundation model feature means owning prompts, an evaluation set to catch regressions, guardrails, and the work of moving before a model reaches its end-of-life date. Neither list is short, and the second is not automatically lighter; it is a different set of things to be responsible for, and it lands on a different sort of person.
What we’ll filter on
- Explanation obligation: does someone have a legal right to be told the principal reasons for one specific decision?
- Labelled history: does the organisation already hold thousands of examples with the outcome recorded?
- Output shape: a label or a number from a fixed set, or open-ended language?
- Volume economics: at real traffic, do per-call cost and latency budgets rule anything out?
- Time and ownership: how quickly must it ship, and who runs it in six months?
The landscape
Three routes are genuinely on the table for a business problem like either of these, and it is worth being precise about what each one requires before comparing them.
A traditional supervised model on Amazon SageMaker AI
You bring labelled historical data, choose an algorithm suited to the shape of the answer, train, evaluate against held-out data, and deploy the resulting artefact to an endpoint. For tabular problems this is usually a gradient-boosted tree or a logistic regression, both of which are small, quick and well understood by the people who audit them. SageMaker Clarify sits alongside it for two jobs: measuring bias in the training data and in the trained model, and producing SHAP attributions for individual predictions. The model registry holds versions, so a decision made in 2024 can be re-run against the artefact that made it.
The work in this route sits upstream of the training job. Without labels there is nothing to train on, and manufacturing labels means paying people to read historical cases and record the outcome.
A purpose-built AWS AI service
Amazon Comprehend, Amazon Textract, Amazon Rekognition, Amazon Transcribe and Amazon Personalize each solve one well-defined problem behind an API, with no model to train for the common cases. Where an ask lands squarely on one of them the choice is usually easy, and this pairing is worked through in more detail in the case for reaching past a foundation model. The wrinkle for the triage feature is that routing to eleven bespoke internal teams is not a general problem. Comprehend would need custom classification, which takes a minimum of fifty labelled training documents per class in the CSV training format. Eleven teams sits well inside the limit of a thousand classes, but that is still five hundred and fifty emails somebody has to label, which puts it back in the same queue as the traditional route.
A foundation model on Amazon Bedrock
A large pre-trained model, called through an API, told what to do in a prompt. No training data, no training job, no endpoint to size. You describe the eleven teams, give a handful of examples, and get a classification back in the same working day. Four things come with that speed: per-token cost at volume, latency in the hundreds of milliseconds, answers that vary between runs, and an explanation you cannot audit feature by feature. Amazon SageMaker JumpStart offers a related shape, where a pre-trained model is deployed to your own endpoint and optionally fine-tuned. You then pay for a running endpoint instead of per token, and the model version stays under your control.
Evaluation
Side by side
| Option | Per-decision explanation | Needs labelled history | Handles open-ended language | Cost and latency at high volume | Ships in a fortnight |
|---|---|---|---|---|---|
| Traditional supervised model on SageMaker AI | ✓ per-feature SHAP attribution | ✗ requires thousands of examples | ✗ tabular and fixed labels | ✓ milliseconds, fractions of a cent | ✗ if the labels do not exist |
| Purpose-built AWS AI service | ✗ confidence scores, not attributions | ✗ for custom classification | ✓ within the task it was built for | ✓ per-request pricing, low latency | ✓ only if the task matches the service |
| Foundation model on Amazon Bedrock | ✗ generated rationale, source traceability at best | ✓ needs none to start | ✓ its strongest ground | ✗ per-token cost, hundreds of milliseconds | ✓ same-week prototype |
Two columns do the separating. The explanation column removes the foundation model from anything a regulator will ask about. The labelled-history column removes the traditional model from anything nobody has ever written the answers down for. Where both columns rule out the same option, the decision is made; where they disagree, the remaining three break the tie.
Where each feature lands
The solution
Credit decisioning goes to a traditional supervised model on SageMaker AI, and the explanation obligation settles it rather than accuracy. The lender has to produce, on demand, the principal reasons for one refusal, seven years after the fact. SHAP attributions produce exactly that, in the same units the refusal letter is written in, and the model artefact is a version in the registry that can be loaded again. A foundation model would very likely predict default well. Its output would not identify what drove one refusal in a form that survives being challenged.
Two operational details ride along. The training data itself needs bias measurement before anyone celebrates the accuracy number. Those pre-training and post-training fairness metrics are published formulas over label counts and confusion-matrix values, so they run in the same pipeline as the SHAP step, and the wider SageMaker division of labour is treated as its own subject in the SageMaker suite’s division of labour. And the retention plan has to cover the model artefact, the training data snapshot and the feature pipeline, not just the decisions, because reproducing a decision needs all three.
Email triage goes to a foundation model on Amazon Bedrock, and the absence of labels settles that one. There is nothing to train on, gathering a training set would take longer than the deadline, and a wrong answer means somebody in the wrong team forwards the email on. Four hundred emails a day at a few hundred tokens each is a rounding error on the monthly bill. Route the low-confidence cases to a human rather than guessing, and log every routing decision along with the correction when a team bounces one back.
That logging is what makes the third row of the diagram possible. After six months the lender owns something it did not have on day one: tens of thousands of emails with the correct team attached, produced as a by-product of running the feature. A small classifier trained on those labels answers in milliseconds at a fraction of the per-call cost, and the foundation model stays in place for the cases the classifier is unsure about. Use the model’s own output as training labels with care, because a classifier trained on them inherits whatever the model got wrong. The human corrections in the log are the part worth trusting most. Have someone check a sample of the rest before it becomes ground truth.
The general shape holds beyond this lender. Where a decision must be explained, is made from structured data, and has been made thousands of times before with the answer recorded, traditional ML models are the defensible choice. Where the input is open-ended language, no labels exist, and a wrong answer is recoverable, foundation models get you to production in days. They answer different questions about what you already own and what you have to prove. The related judgement of whether to reach for generative AI at all is worked through in the harder version of this call, and the argument for trying the simplest model first is made in the case for the unglamorous baseline.
Worked example
Two more asks arrive at the same lender a month later. Both go through the same gates.
Blocking fraudulent card transactions
Nine years of transactions, every one of them flagged or not flagged by the fraud team afterwards, so labels exist in the millions. The answer is a score between zero and one. The latency budget is forty milliseconds, because the decision happens while the card is at the terminal. Customers whose transactions are blocked complain, and the complaints team needs to say why.
Every gate points the same way. Labels exist, the output is a number, the budget rules out anything token-priced, and the explanation obligation rules out anything without attribution. This is a traditional supervised model, and the interesting work is in the imbalanced-classes problem rather than in the choice of route.
Answering agent questions from the policy handbook
Four hundred pages of product terms, updated quarterly. Agents ask questions in their own words during calls. Nobody has ever written down which passage answers which question, so there are no labels. The answer is a paragraph of English, not a label. A wrong answer is caught by the agent reading it before they say it aloud.
No labels, open-ended output, a human between the model and the customer, and no per-decision explanation obligation because the agent owns what they say. This is a foundation model over the handbook, and the design work goes into retrieval and into showing the agent which passage the answer came from.
What’s worth remembering
- A per-decision explanation obligation is a hard filter: traditional ML models give per-feature attribution through SHAP values, while a foundation model returns a generated rationale and, when grounded, traceability to its sources.
- Labelled history is the other hard filter: a traditional model needs thousands of labelled examples, and foundation models (FMs) need none to start.
- Structured inputs with a label or a number as the answer point at a traditional model; open-ended language in or out points at a foundation model.
- At high volume the economics separate, because a small trained model answers in milliseconds from an endpoint you are already paying for, while a foundation model is charged per input token and per output token.
- Operational constraints decide the ties: one route means owning labelling, training and retraining, the other means owning prompts, evaluation and somebody else’s model lifecycle.
- The two routes combine, because a foundation model shipped now produces the labelled data that trains a smaller, cheaper model later, provided the human corrections are kept and a sample of its own labels is checked.