The situation
A company that builds record-keeping software for medical clinics is adding three generative features in the same release. The first summarises a clinician’s free-text consultation notes into a structured handover for the next appointment. The second turns a completed treatment plan into a plain-language explanation the patient can read at home. The third tags the photographs patients upload to their record (a rash, a healing wound, a medication packet) so those images can be found again by description rather than by date.
The numbers differ as much as the jobs do. Around forty thousand consultations a day produce notes, and the summary is written minutes after the clinician closes the record, so nobody is waiting on it. The patient explanation is generated while the patient is still at the reception desk, so a person is watching a spinner. Photo tagging runs on about nine thousand uploads a day and can happen any time in the following hour.
The proposal on the table is to standardise on a single model for all three, and the argument for it starts outside engineering. One supplier to put in front of the clinic’s compliance officer, one set of provider terms to read, one line on the invoice, one integration to keep patched. Pick the most capable thing in the Amazon Bedrock catalogue, the reasoning goes, and every feature is covered by something that can certainly do the job. The governance half of that argument is worth taking seriously. The rest of it has a bill nobody has added up.
What actually matters
Start with what goes in and what comes out, because that alone rules out most of the catalogue for two of the three features. Text in and text out covers the summary and the explanation. Photo tagging needs a model that can look at an image and describe it in words, which is a different capability from generating an image from a description, and both are different again from turning text into a vector for search. Getting the modality wrong is not a quality problem that better prompting fixes; it is a model that cannot do the job at all.
Then quality, which is worth stating as a threshold rather than a ranking. Each of these three features has a level below which it is unusable and above which extra headroom buys nothing anyone can see. A handover summary that misses a medication change is worthless; a handover summary that is beautifully written adds no clinical value over one that is accurate and complete and nothing more. Ask per feature which models clear that line. The artefact that answers it is thirty or forty real examples with a known good answer, scored the same way for every candidate. Without that set, “most capable” means whatever a vendor benchmark measured, on a task that is not yours.
Cost and speed then separate the models that all cleared the line, and they pull the opposite way from quality. Generative models bill per token. A feature’s bill is its volume multiplied by its prompt and response length multiplied by the model’s rate, and those rates span more than an order of magnitude across a catalogue. Forty thousand summaries a day on a flagship model is a very different monthly figure from forty thousand on a small one, for a job whose quality bar the small one may well clear. Speed splits by who is waiting. An overnight job can absorb seconds that a patient at a reception desk cannot.
Last come the things that hold regardless of how good or cheap a model is. It has to be available in the Region where you are allowed to process the data, and catalogue availability varies by Region. The provider’s terms have to permit the kind of data you are sending. The context window has to fit your longest input, and the output limit your longest answer. In a clinical setting these are gates rather than preferences. A model that fails one of them is off the shortlist, however well it scored.
What we’ll filter on
- Model types. Does the job need text in and text out, an image understood and described, an image generated, or text turned into a vector? Multi-modal models take more than one kind of input; single-modality models do not.
- Capabilities and performance requirements. What the model has to be good at (summarising, following a format, plain-language rewriting, describing an image) and the measured level it has to reach on your own examples before it is a candidate.
- Cost. The bill at the volume this feature actually runs at, priced per token rather than per call.
- Latency. The wait a person experiences, which matters for an interactive feature and barely matters for a background one.
- Constraints and model complexity. Context-window and output limits, Regional availability, quota, and how much model the task needs; a larger, more complex model brings reasoning machinery a simple extraction task never uses and still pays for.
- Compliance. Which Regions the model can run in, what the provider’s terms allow you to send, and whether the workload needs an isolated deployment rather than a shared endpoint.
The landscape
The catalogue is easier to weigh as a handful of classes than as a list of model names, because names and versions turn over every few months while the classes have held still. A feature matched to the right class survives a refresh as a version bump rather than a redesign. The per-model detail belongs to the developer-level walk through the Bedrock catalogue, which goes family by family. What a business weighing needs from the catalogue is narrower: where the classes sit on capability, on price, and on what they commit you to.
Capability comes first because it is absolute. Four things a model might do are genuinely separate. Take text and answer in text, which covers the summary and the explanation. Take an image and answer in words, which covers the photo tagger. Take a description and draw a picture. Take a passage and return a vector for search. Budget moves nothing between those four. The Amazon Nova family spans several of them at once, which is convenient and also the reason teams assume a family name is a capability guarantee; Nova Micro is text-only, while Nova Lite and Nova Pro accept images and documents alongside text.
Price and wait form a ladder inside the text classes, and the rungs are far apart. A small, fast tier bills the least per token and returns a first token soonest, and gives up ground on long multi-step reasoning and on holding a complicated instruction set together over a long output. A mid-range tier follows a detailed format reliably and handles documents of real length for a fraction of the flagship rate. A flagship tier repays its price on ambiguous instructions, code, and analysis holding several constraints at once, and repays nothing at all on extraction. End to end the per-token rates span more than an order of magnitude, which is why the ladder is a cost decision before it is a quality one.
The commitments are the part a technical comparison tends to skip. Whether the weights are open or proprietary changes your hosting and licensing options rather than your shortlist, which the open-weight and proprietary comparison works through. How you buy capacity, on demand per token or as reserved throughput, is a separate decision that the token-pricing walkthrough covers. And the catalogue is not identical in every Region, provider terms differ on what data you may send, and some workloads need an isolated deployment rather than a shared endpoint. Those three vary by model, and in a clinical setting they decide eligibility before anything else does.
Evaluation
Side by side
| Model class | Model type, in and out | What it clears without headroom to spare | Cost at this scenario’s volumes | Latency a person would notice | Constraint that bites first | Complexity paid for |
|---|---|---|---|---|---|---|
| Dedicated embedding | text or an image in, a vector out | nothing readable; retrieval only | very low | none | vector dimensions and index size | low |
| Image generation | text in, a picture out | imagery, not description | per image rather than per token | seconds | content filters and output size | not applicable |
| Small fast text (Nova Micro tier) | text in, text out | extraction, classification, reformatting | lowest | none | the shortest context windows in the catalogue | low |
| Mid-range general text (Nova Lite, Nova Pro) | text, images or documents in, text out | summarising and rewriting to a format | moderate | slight | output ceiling on long answers | medium |
| Multi-modal | text and an image in, text out | describing what is visible in a picture | moderate to high | slight | image size and count per request | medium to high |
| Flagship reasoning | text and often images in, text out | ambiguous, multi-step work | highest | real | Regional spread, thinnest here | high |
Read that table in two passes. The first two columns answer with a yes or a no against the job in front of you, and a no there is the end of it: a model that cannot take an image will not learn to, and a model that hands back a vector has nothing to say to a patient. Whatever survives goes into the second pass, where cost, latency, constraints and complexity are all quantities, so a feature can trade one against another and argue about where the line sits. Teams that start from a price list or a benchmark leaderboard run the passes in the other order and end up defending a model that was never eligible.
Which job lands where
Three features, three classes, and the single model that was going to serve all three serves none of them. That is the usual shape once the filters run separately, and it is why the eight factors get applied per feature rather than per organisation.
The solution
Route rather than standardise. Each feature gets the smallest model that clears its own quality bar, measured on its own examples, and the three sit on three different models behind one internal interface. The governance saving the standardisation argument was reaching for does not vanish; it moves. Three provider assessments instead of one is real work, and it is worth costing honestly against a bill that would otherwise run several times higher every month for output nobody can tell apart.
Photo tagging goes to a multi-modal model, because nothing else in the catalogue can do it. Within that class, pick the cheaper tier first: the job is describing what is visibly in a photograph, not diagnosing from it, and that distinction is worth writing into the prompt and the product copy. Nine thousand uploads a day, each with an hour of slack behind it, makes latency close to free here, so the filter that decides is cost.
The note summary goes to a small fast text model. Forty thousand a day is the volume that makes per-token price the dominant term in the bill, and turning a clinician’s notes into a fixed handover structure is extraction and reformatting rather than reasoning. Run the scored example set against the small tier before committing, and if it misses medication changes, step up one tier and run it again. Stepping up one tier at a time, with a measurement between each step, is how you find the floor instead of guessing at it.
The patient explanation goes to a mid-range general text model. Rewriting clinical language into something a worried person can follow is the hardest of the three linguistically, a person is waiting on it, and the volume is a fraction of the summary volume, so the cost filter has less to say. Test the small tier anyway; if it clears the bar the wait gets shorter as well as cheaper. Where a feature has a mix of easy and hard cases, routing between a cheap and a capable model is the pattern that splits them, and the end-to-end latency breakdown shows how much of the wait is the model and how much is everything around it.
Compliance runs across all three and is checked before any of it ships. Confirm each chosen model is available in the Region the clinic data may be processed in. The catalogue is not identical in every Region, and a model unavailable where you need it is not a candidate at all. Confirm the provider’s terms permit the data you intend to send. Confirm what is logged, where those logs live, and who can read them. Where the workload needs an isolated deployment rather than a shared on-demand endpoint, that changes the shortlist. Establish it at the start, not after a model has been chosen and integrated.
Worked example
Take the patient explanation and run the six filters in order.
Model types removes image generation and embedding models immediately: text goes in, text comes out, and no picture or vector is involved. Multi-modal models stay eligible, since they take text as well, but nothing about this job needs the extra input type.
Capabilities and performance requirements are where the example set does its work. Forty real treatment plans, each with an explanation a clinician has approved, scored for accuracy, reading level, and whether any instruction was dropped. Run every candidate against the same forty. The small tier clears accuracy but drops a caveat in four of them; two mid-range models clear all three measures; the flagship clears them too, with no visible difference from the mid-range pair.
Cost now separates the three that passed. At this feature’s volume the mid-range models cost a few hundred dollars a month and the flagship several times that, for output that scored the same. The flagship leaves the shortlist here, on price and nothing else.
Latency decides between the two survivors, because somebody is standing at a desk. Measure time to first token and total response time on your own prompt lengths, not on the published figures, and take the faster one.
Constraints and model complexity are the check before you commit. The longest treatment plan plus its instructions has to fit the context window, and the longest explanation has to fit the output limit, with room to spare. The mid-range winner has the model complexity this job needs and no more. The flagship’s extra reasoning capacity had nothing to do here, which is why it made no difference to the scores and every difference to the bill.
Compliance is the last gate and the one with a veto. The chosen model has to be available in the Region the data is confined to, and the provider’s terms have to cover clinical text. If it fails either, go back to the second-place model from the latency step, which is the reason for keeping a ranked shortlist rather than a single winner.
What’s worth remembering
- The eight factors (model types, performance requirements, capabilities, constraints, compliance, cost, latency, and model complexity) run per feature rather than per organisation, so three features in one release can land on three different models.
- Model type eliminates and the numbers rank, so filter on what a model can do at all before comparing what it costs, and never the other way around.
- Multi-modal models take more than one kind of input and answer in text, which is a different job from generating an image and different again from producing a vector.
- More model complexity buys reasoning capacity a simple extraction task never uses, so start at the smallest tier, such as the lower Amazon Nova models, and step up only when a scored example set says you must.
- Cost is volume multiplied by tokens multiplied by rate, and latency bites only where a person is waiting, so a high-volume background job and a low-volume interactive one land at opposite ends of the catalogue.
- Compliance is a gate rather than a preference, because Regional availability, the provider’s terms for your data, and any need for an isolated deployment can remove the best-scoring model; check all three before the integration is built.