The situation
A product team has a backlog of features that all sound like AI work, and the default plan for every one of them is to call a large language model on Amazon Bedrock. There is a form that needs a postcode validated. There is a stream of scanned invoices to pull totals out of. There is a support inbox that needs each message routed to the right queue. There is a photo-upload flow that should reject pictures with no product in them. And there is a genuinely open-ended one: a knowledge assistant that answers staff questions by reasoning over a pile of internal documents.
The instinct is to reach for the same generative model for all five, because it can plausibly do all five. Ask it to validate the postcode, ask it to read the invoice, ask it to route the ticket, ask it to describe the photo, ask it to answer the question. One integration, one mental model, one bill. The bill is the first thing that gives the team pause: five features all making per-token calls to a frontier model, several of them on inputs that arrive thousands of times a day. The second is a near-miss in testing, where the postcode validator confidently accepted a malformed code because the model felt agreeable that afternoon.
The question underneath all five is the same. A generative model can do the task, but is it the right tool, or is there something cheaper, faster, and more predictable that fits the shape of the work better?
What actually matters
A generative LLM is not a general upgrade over every other kind of software. It is a particular tool with a particular profile: extraordinarily flexible on open-ended language, and in exchange nondeterministic, priced per token, slow relative to a lookup, prone to producing confident wrong answers, and expensive to evaluate because there is rarely a single correct output to diff against. Every one of those costs is worth paying when the flexibility is the point. None of them is worth paying when a narrower tool would settle the task outright.
The dividing line that decides the most is whether the task is well-defined or open-ended. A well-defined task has a knowable correct answer and a describable rule for reaching it: is this string a valid postcode, does this transaction exceed a threshold, which of six queues does this ticket belong in. Tasks like that have a right answer you can test against, and the closer you get to a crisp specification the more a deterministic rule, a lookup table, or a trained classifier will beat a generative model on cost, speed, and reliability at once. An open-ended task has no single correct output: summarise this contract, draft a reply in this tone, answer this question from these documents. That is where a generative model earns its place, because the space of acceptable outputs is too large and too fuzzy for anything narrower to cover.
The second axis is tolerance for error, and specifically the shape of the errors. A deterministic rule fails predictably: it is wrong in exactly the cases the rule doesn’t cover, and you can enumerate them. A generative model fails unpredictably and often invisibly, producing a fluent answer that is simply untrue, which is the failure mode people mean by hallucination. If a wrong answer is cheap to catch and cheap to fix, the unpredictability is tolerable. If a wrong answer flows straight into a downstream system, a payment, or a compliance record, the burden of catching it, through evaluation, guardrails, and human review, lands back on you and has to be counted as part of the model’s cost.
The third is the operational profile: latency, throughput, and unit cost. A regex or a hash lookup answers in microseconds for effectively nothing. A purpose-built AWS AI service answers in tens to hundreds of milliseconds at a fixed, published per-unit price. A frontier LLM answers in hundreds of milliseconds to seconds and charges per token in and out, so a task running at high volume on long inputs is precisely where the generative option is least competitive on cost and worst on latency. Volume and input length turn a rounding-error difference per call into the line item that reshapes the budget.
The fourth is validation and change control. A rule is readable, reviewable, and testable: you can prove what it does. A classic classifier has a measurable accuracy on a labelled test set. A generative feature has to be evaluated on a corpus of examples with a scoring method you build yourself, re-run whenever the prompt or the model version changes, and defended with guardrails against the outputs you never want. That evaluation and guardrail burden is real engineering work, and it is the tax the model charges for its flexibility.
Put together, these say something simple: reach for a generative model when the task is genuinely open-ended and the flexibility is worth the unpredictability, the token cost, and the evaluation burden. Where the task is well-defined, a narrower tool is usually cheaper, faster, and easier to trust, and the model is the wrong default.
What we’ll filter on
- Task definition, is there a knowable correct answer and a describable rule, or is the output open-ended?
- Determinism needed, does the same input have to produce the same output every time?
- Error tolerance, what does a wrong answer cost, and how easily is it caught before it does damage?
- Volume and latency, how many calls, how long are the inputs, and how fast must the answer come back?
- Validation burden, can correctness be tested cheaply, or does it need a bespoke evaluation and guardrail effort?
- Flexibility required, does the task genuinely need open-ended language understanding, or is that just the convenient framing?
The alternatives landscape
Deterministic rules and lookups. A regular expression, a validation function, a hash-map lookup, a threshold comparison. For anything with a crisp specification, a postcode format, a currency threshold, a known set of routing keywords, this is the fastest, cheapest, and most reliable option there is, and it is fully testable. The failure mode is brittleness: a rule only covers the cases you wrote it for, and messy real-world input that doesn’t fit the pattern falls through. When the specification really is knowable, that trade is almost always worth it.
Traditional search and information retrieval. Keyword search, an inverted index, Amazon OpenSearch, or a database query. When the job is finding the right existing document or record rather than composing a new answer, plain search is cheaper and more predictable than asking a model to recall or generate. It also underpins the open-ended case: retrieval feeds the documents to a generative model in a retrieval-augmented setup rather than the model being asked to remember them, so search and generation are often partners, not rivals.
Classic machine learning classifiers. A model trained on labelled examples to sort inputs into a fixed set of categories: sentiment, spam, ticket routing, fraud flags. Amazon SageMaker trains and hosts these, and for a well-defined classification task with training data available, a purpose-trained classifier is cheaper per call, lower latency, deterministic for a given model version, and measurable against a test set. It needs labelled data and retraining as the world shifts, which is its main cost.
Purpose-built AWS AI services. Managed models for specific, common tasks, offered at a fixed per-unit price with no model to train. Amazon Comprehend for entity extraction, sentiment, and language detection over text. Amazon Textract for pulling text, forms, and tables out of scanned documents. Amazon Rekognition for object, scene, face, and moderation detection in images and video. Amazon Transcribe for speech to text, Amazon Translate for language translation. For the task each was built for, these beat a general LLM on price, latency, and consistency, and they return structured, confidence-scored output you can threshold on. They only fit inside their designed scope; push past it and you are back to a general model.
Plain software. Sometimes the honest answer is that no model of any kind is needed: a calculation, a state machine, a database join, a bit of business logic. If the task is a computation with a known procedure, code is the tool, and reaching for AI at all is the over-engineering.
Generative large language models. A frontier model on Amazon Bedrock, invoked directly or through an agent. This is the tool for open-ended language: summarising messy text, drafting and rewriting, extracting from genuinely unstructured input that no fixed schema anticipates, holding a conversation, and reasoning over documents to answer questions there is no lookup for. It buys flexibility no narrower tool can match, and it charges for it in nondeterminism, per-token cost, latency, and the evaluation and guardrail work needed to trust the output.
Side by side
| Approach | Best for | Deterministic | Handles open-ended input | Per-call cost | Validation ease |
|---|---|---|---|---|---|
| Rule / lookup | Crisp, knowable specifications | ✓ | ✗ | Lowest | Fully testable |
| Traditional search | Finding existing documents or records | ✓ | Partial | Low | Testable |
| Classic ML classifier | Well-defined classification with labels | ✓ (per version) | ✗ | Low | Test-set accuracy |
| Purpose-built AWS service | The specific task it was built for | ✓ (per version) | Within scope | Fixed per unit | Confidence scores |
| Plain software | Known computations and business logic | ✓ | ✗ | Lowest | Fully testable |
| Generative LLM | Open-ended language and reasoning | ✗ | ✓ | Highest (per token) | Bespoke evaluation |
Reading the table against the five features: the postcode check is a rule, the invoice totals are Textract, the ticket routing is Comprehend or a classic classifier, the photo check is Rekognition, and only the knowledge assistant is genuinely a generative model. Four of the five never needed an LLM at all.
The picks in depth
The postcode validator is the plain-rule case, and it is the one where using a model is actively worse. A postcode has a knowable format; a regular expression or a validation library decides it in microseconds, for nothing, deterministically, and with a test suite that proves exactly which inputs pass. Handing that to a generative model buys nondeterminism on a task that must never be nondeterministic, pays per token to answer a question a regex answers for free, and introduces the failure the team already saw in testing, a confident acceptance of an invalid code. When the specification is this crisp, the model is the wrong tool by every measure that matters.
The invoice extraction is the purpose-built-service case. Pulling totals, dates, and line items out of scanned documents is exactly what Amazon Textract exists for: it returns the fields with confidence scores at a fixed per-page price, far cheaper and more consistent than feeding page images to a general model and hoping the numbers come back right. Where the layout is truly chaotic and no structured extractor copes, a generative model becomes a reasonable fallback, but that is the exception to reach for after Textract, not the default to start from.
The ticket routing is the classifier case. Sorting each message into one of six known queues is a well-defined classification problem, and if there is labelled history, either Amazon Comprehend’s custom classification or a classic model trained on SageMaker will route it cheaper, faster, and more predictably than an LLM, with an accuracy number you can measure and watch. A generative model can classify too, but paying per token and accepting nondeterminism to do a job a purpose-trained classifier does better is the pattern this whole exercise is meant to catch.
The photo check is the vision-service case. Deciding whether an uploaded image actually contains a product is object and scene detection, which Amazon Rekognition does at a fixed price with confidence thresholds you can tune. It is narrower and cheaper than a multimodal LLM and returns exactly the structured signal the flow needs.
The knowledge assistant is the one genuine generative case, and it is worth seeing why it clears the bar the others didn’t. There is no fixed set of answers, no rule that maps a question to a response, and no lookup that composes a coherent explanation from several documents at once. The output is open-ended language, the flexibility is the point, and the task genuinely needs reasoning over unstructured text. That justifies paying the token cost, accepting the nondeterminism, and taking on the evaluation and guardrail work, because nothing narrower can do it. Notice the shape of the good decision: the model earns its place not because it can do the task but because everything cheaper cannot.
The routing above is the whole decision compressed: a task only reaches the generative box after a knowable-answer test and a purpose-built-fit test have both failed to place it somewhere cheaper. Defaulting to the model inverts that, starting at the most expensive, least predictable option and never asking whether anything narrower would have done.
A worked example: the invoice line, before and after
The task is pulling the total from a stream of scanned supplier invoices, roughly forty thousand a month, feeding an accounts-payable system that pays against the figure.
Before. Each invoice page is sent as an image to a multimodal model on Bedrock with a prompt: read this invoice and return the total as a number. It works most of the time. It also, on a handful of awkward scans a week, returns the subtotal instead of the grand total, or transposes two digits, or reads a faint figure confidently wrong. Because the answer is a fluent number with no confidence attached, nothing downstream flags it; the wrong total flows straight into a payment. On top of that, forty thousand multimodal calls a month on full-page images is a meaningful bill, and every prompt tweak means re-checking the whole thing by hand because there is no test set, only spot checks.
After. Amazon Textract’s expense analysis is built for exactly this: it extracts invoice fields, including the total, and returns each with a confidence score at a fixed per-page price well below a multimodal model call. The confidence score is the part that changes the risk profile. Anything above the threshold flows straight through; anything below is routed to a person before payment, so the awkward scans that used to become silent wrong payments now become a small review queue instead. The cost drops, the latency drops, the output is structured and thresholdable rather than a bare number, and correctness is measurable against a labelled set of invoices rather than eyeballed. The generative model was doing an open-ended reading job on a task that was never open-ended; the total was always sitting in a known field, waiting for the tool built to read it.
What’s worth remembering
- A generative model is the most flexible tool available and the most expensive, least predictable, and hardest to validate; match the tool to the task instead of defaulting to the model.
- Well-defined tasks with a knowable correct answer belong to rules, lookups, classifiers, or purpose-built services, which are cheaper, faster, deterministic, and testable.
- A generative LLM earns its place on genuinely open-ended work: summarisation, drafting, extraction from unanticipated unstructured text, conversation, and reasoning over documents.
- Reach for the model because nothing narrower can do the task, not because the model can also do it; being capable is not the same as being the right fit.
- Deterministic tools fail predictably in cases you can enumerate; generative models fail invisibly with fluent wrong answers, so weigh what a wrong answer costs and how easily it is caught.
- Purpose-built AWS services fit specific shapes: Textract for documents, Comprehend for text analysis, Rekognition for images, Transcribe and Translate for speech and language.
- High volume and long inputs are where the generative option is least competitive, because per-token cost and latency scale with exactly those.
- The token bill is only part of the cost; nondeterminism, hallucination risk, evaluation effort, and guardrails are the tax the model charges for its flexibility.
- Confidence scores from a purpose-built service turn silent wrong answers into a review queue; a bare generated answer offers nothing to threshold on.
- Start the decision from the task and its tolerance for error, and let it flow to a generative model only after the cheaper, more predictable options have genuinely failed to fit.