The situation
A product team is standing up four features on Amazon Bedrock. The first is a high-volume ticket classifier that tags each incoming support message with one of eight labels; it runs millions of times a month and every millisecond and fraction of a cent shows up in the bill. The second is a retrieval assistant that answers questions over the company handbook, which means turning documents and queries into vectors so the closest passages can be found. The third is a hard-reasoning helper that untangles multi-step policy questions where a wrong answer is expensive. The fourth is a marketing tool that generates product imagery from a text brief.
Right now all four route to the same flagship chat model, because that was the one someone enabled first and it clearly works. The classifier is paying flagship prices to pick between eight labels. The retrieval feature is asking a chat model to “find similar text” when it should be producing embeddings. The image feature does not work at all, because a text model cannot draw. The bill is large and the latency is worse than it needs to be on the two features that run most often.
Nobody wants to benchmark every model in the catalogue by hand. The question underneath all four features is the same: given the shape of this workload, which class of model does it actually need, and what is the smallest one that clears the bar?
What actually matters
The Bedrock catalogue is not a quality ladder where the top model is the right answer and everything below it is a compromise. It is a spread of model classes built for different jobs, and the first decision is not “which model” but “which kind of model”. Picking the flagship chat model for an embeddings job is not paying too much for quality; it is using the wrong tool entirely.
The dividing line that decides the most is modality. What goes in and what comes out. A text-in, text-out task, an image-generation task, a task that needs to read an image or a document as well as text, and a task that produces vectors for retrieval are four different capabilities, and only some models offer each. Amazon’s own Nova family spans several of these on its own: Nova Micro is text-only and built for speed, Nova Lite and Nova Pro accept multimodal input, Nova Canvas generates images, and Nova Reel generates video. Anthropic’s Claude, Meta’s Llama, Mistral, Cohere, and AI21 cover text and, for several of them, multimodal input. Amazon Titan and Cohere both offer dedicated embeddings models. Stability AI and Amazon’s Nova Canvas cover image generation. Match modality first, because nothing else matters if the model cannot produce the shape of output the task needs.
Once modality is settled, the next axis is the reasoning difficulty of the task, because that is what justifies model size. A classifier picking one of eight labels, a sentiment call, a short extraction: these are easy judgements, and a small, fast, distilled model such as Nova Micro or Nova Lite, or the smaller tier of another provider’s line-up, clears them at a fraction of the cost and latency of a flagship. A multi-step policy deduction, a tricky synthesis, a task where a subtle mistake is costly: these are where a large model such as Nova Pro or a flagship Claude earns its price, because the extra capability actually changes the answer. Spending flagship tokens on the classifier buys nothing; spending small-model tokens on the hard reasoner buys a wrong answer.
Cost and latency move together and both track model size. Smaller and distilled models are cheaper per token and answer faster; larger models cost more on both the input and the output side and take longer to respond. Because pricing is per token split between input and output, a task with long inputs (a big retrieved context) or long outputs (verbose generation) feels the price of a large model much more sharply than a short classification does. High call volume multiplies every one of those fractions, which is why the classifier’s model choice matters more to the bill than the rarely-used hard reasoner’s does.
Context-window size is its own axis. If the task stuffs a large document, a long conversation, or a big pile of retrieved passages into the prompt, the model has to have a window big enough to hold it, and the models differ widely here. A short classification needs almost none; a long-document summariser or a retrieval feature with generous context needs one of the larger windows. Do not pay for a huge window a short task never fills, and do not pick a small-window model for a task that routinely overflows it.
Two more axes decide the edges. Whether the model supports fine-tuning or customisation matters if the task needs the model to learn a house style or a domain vocabulary that prompting alone cannot pin down; only some models in the catalogue can be customised, so if that is a requirement it narrows the field early. And region availability plus model access is an operational gate that trips teams up: not every model is offered in every region, and even an available model has to be explicitly enabled for the account before any call to it succeeds. A model that is perfect on paper but not enabled in your region is not a choice you can make today.
What we’ll filter on
- Modality, what goes in and what comes out? Text, multimodal input, image or video generation, or embeddings.
- Reasoning difficulty, is this an easy snap judgement or a hard multi-step problem that justifies a large model?
- Cost and latency budget, how sensitive is this workload to per-token price and response time, especially at volume?
- Context-window size, does the task feed in long documents or large retrieved context, or almost nothing?
- Customisation, does it need fine-tuning to learn a style or vocabulary, or will prompting do?
- Region and access, is the model offered in the target region and enabled for the account?
The model landscape
Amazon Nova (Micro, Lite, Pro). Amazon’s own text and multimodal family, priced and tuned as a tiered line-up. Nova Micro is text-only and built for the cheapest, fastest responses, which suits high-volume classification and extraction. Nova Lite and Nova Pro accept multimodal input (text plus images or documents), with Lite as the balanced mid-tier and Pro as the most capable of the three for harder reasoning. The tiering is the point: pick the smallest Nova that clears the task and step up only when quality demands it.
Amazon Nova Canvas and Nova Reel. The generation side of the Nova family. Canvas produces images from text prompts; Reel produces short video. These are the reach-for models when the output is a picture or a clip rather than text, and they sit alongside Stability AI’s image models in that part of the catalogue.
Amazon Titan (text and embeddings). Amazon’s earlier first-party line. The Titan embeddings models turn text into vectors for retrieval and semantic search, which is a different job from chat and the right tool for the retrieval feature. Titan text models handle general text generation.
Anthropic Claude. A strong general-purpose text and multimodal-input family, frequently the pick for hard reasoning, nuanced writing, and tasks where output quality carries the feature. Available across a range of sizes so you can trade capability against cost within the same provider.
Meta Llama. Open-weight text (and, for newer versions, multimodal-input) models offered on Bedrock across several sizes, a common choice when a team wants a capable general model with a different cost profile or licensing posture.
Mistral. Efficient text models spanning small, fast options through larger, more capable ones, often chosen for a good quality-per-cost balance on general text tasks.
Cohere. Text generation plus a well-regarded embeddings line, which makes Cohere another candidate for the retrieval feature alongside Titan embeddings.
AI21. Text models for general generation and language tasks, another option in the general-purpose text tier.
Stability AI. Image-generation models for the text-to-image job, the third-party counterpart to Nova Canvas.
Across all of these, two operational choices sit on top of the model pick. On-demand inference bills per token with no commitment and is the default for variable or spiky traffic; Provisioned Throughput reserves capacity for steady, high-volume, latency-sensitive workloads and can lower unit cost and stabilise latency when the volume justifies the reservation. And cross-region inference profiles let a request be served from one of several regions to raise available throughput and smooth capacity, which helps a high-volume feature scale without hitting a single region’s limits.
Side by side
| Model class | Modality | Reasoning tier | Cost / latency | Context need | Fine-tuning | Typical fit |
|---|---|---|---|---|---|---|
| Nova Micro | Text in, text out | Easy | Lowest | Small | Some models | High-volume classify / extract |
| Nova Lite | Multimodal in, text out | Easy–medium | Low | Medium | Some models | Balanced everyday text and vision |
| Nova Pro | Multimodal in, text out | Hard | Higher | Large | Some models | Harder reasoning with images |
| Claude (large) | Multimodal in, text out | Hard | Higher | Large | Varies | Nuanced writing, hard reasoning |
| Llama / Mistral / AI21 | Text (some multimodal) in | Easy–hard by size | Varies by size | Varies | Varies | General text, cost-balanced |
| Titan / Cohere embeddings | Text in, vectors out | ✗ (not generative) | Low | N/A | Some | Retrieval and semantic search |
| Nova Canvas / Stability AI | Text in, image out | ✗ | Per image | N/A | ✗ | Image generation |
| Nova Reel | Text in, video out | ✗ | Per clip | N/A | ✗ | Short video generation |
Reading the table against the four features: the classifier wants Nova Micro or a small text tier; the retrieval assistant wants an embeddings model (Titan or Cohere), not a chat model at all; the hard reasoner wants Nova Pro or a large Claude; the marketing tool wants Nova Canvas or Stability AI. One flagship chat model was the wrong answer for three of the four.
Routing a workload to a model class
The decision is a small cascade: settle modality, then, for the text branch, let reasoning difficulty and volume choose the size.
The gates after the size pick are the ones teams forget: does the chosen model’s context window hold the largest realistic input, does the task need fine-tuning that only some models offer, and is the model actually offered in the target region and enabled for the account? Any one of those can send you back a step.
The picks in depth
The classifier is the clearest saving. Eight labels, one short input, one short output, running millions of times a month: this is an easy judgement at enormous volume, which is exactly the profile Nova Micro is built for. Moving it off the flagship and onto the cheapest text tier cuts the per-call price and the latency at the same time, and because the task never needed deep reasoning, accuracy holds. At this volume the model choice is the single biggest lever on the bill, far more than on any low-traffic feature. If the traffic is steady and heavy enough, this is also the feature where Provisioned Throughput and a cross-region inference profile start to pay for themselves, reserving capacity and spreading load so latency stays flat under peak.
The retrieval assistant needs a change of model class, not a smaller chat model. Finding the passages closest in meaning to a question is an embeddings job: an embeddings model such as Titan or Cohere turns documents and queries into vectors, and the nearest vectors are the relevant passages. A chat model asked to “find similar text” is doing the wrong job expensively. The embeddings model produces the vectors that fill the index; a separate generative model then writes the answer from the retrieved passages, and that generator is where the earlier modality-and-difficulty cascade runs again. This split between the embeddings model and the answer model is the heart of a retrieval-augmented setup on Bedrock.
The hard reasoner is the one feature that genuinely earns a large model. Multi-step policy questions where a subtle error is costly are where Nova Pro or a large Claude changes the answer, not just the token count, and where a bigger context window matters if the policy documents are long. Because it runs far less often than the classifier, its higher per-call cost barely moves the total bill, so this is the right place to spend. The discipline still applies in reverse: start from the capable model here because the task demands it, but do not let that same reflex push the flagship onto the classifier.
The marketing tool simply needs the right modality. A text model cannot generate an image, so this routes to Nova Canvas or Stability AI, billed per image rather than per token. If short video is ever on the brief, Nova Reel covers that branch. The only real gates here are region availability and enabling model access, because generation models are not offered everywhere and, like every model on Bedrock, have to be explicitly enabled before the first call works.
A worked example: sizing the classifier against the reasoner
Put the two text features side by side and the sizing logic falls out. The classifier receives a short message, returns one of eight labels, and runs, say, three million times a month. The reasoner receives a policy question with a few supporting paragraphs, returns a careful multi-paragraph answer, and runs a few thousand times a month.
For the classifier, the input and output are both tiny, so per-token price dominated by volume is the whole story. A small model such as Nova Micro at the cheapest tier, multiplied across three million short calls, is dramatically less than the same calls on a flagship, and the answers are just as good because eight-way tagging is not a reasoning problem. The context window can be small; there is nothing long to hold. On-demand is fine to start, and if the volume stays high and steady, Provisioned Throughput plus a cross-region profile keeps latency flat and unit cost down.
For the reasoner, the input carries real context and the output is long and must be right, so a large model such as Nova Pro or a large Claude is the correct spend, and a generous context window matters because the policy paragraphs have to fit. The per-call cost is far higher, but a few thousand calls a month against millions for the classifier means the reasoner is a rounding error on the bill. Spending big here and small there is not inconsistency; it is matching each model to the shape of its own workload. The mistake the team started with was the opposite: one big model for both, overpaying massively on the feature that runs constantly to avoid re-deciding on the feature that barely runs at all.
What’s worth remembering
- Decide the model class before the model: modality first (text, multimodal input, image, video, or embeddings), because the wrong class cannot produce the right output at any price.
- The Nova family alone spans several classes: Micro for fast text, Lite and Pro for multimodal input, Canvas for images, Reel for video; step up the tier only when quality demands it.
- Retrieval is an embeddings job, not a chat job; use a Titan or Cohere embeddings model to build the index, then a separate generative model to write the answer.
- Pick the smallest model that clears the quality bar; on easy, high-volume tasks a small distilled model such as Nova Micro is both cheaper and faster, and accuracy holds because the task never needed reasoning.
- Reserve large models such as Nova Pro or a flagship Claude for genuinely hard reasoning, where the extra capability changes the answer rather than just the bill.
- Cost and latency track model size, and pricing is per token split between input and output, so long-context or verbose tasks feel a big model’s price far more sharply than short ones do.
- Match the context window to the largest realistic input; do not pay for a huge window a short task never fills, or pick a small-window model for a task that overflows it.
- If the task needs fine-tuning to learn a style or vocabulary, check early, because only some catalogue models support customisation.
- Region availability and model access are gates, not details: a model has to be offered in your region and explicitly enabled for the account before any call succeeds.
- On-demand suits variable traffic; Provisioned Throughput and cross-region inference profiles pay off for steady, high-volume, latency-sensitive features once the volume justifies reserving capacity.