The situation
A product team has a backlog of features that all sound like AI work, and the default plan for every one of them is to call a large language model on Amazon Bedrock. There is a form that needs a postcode validated. There is a stream of scanned invoices to pull totals out of. There is a support inbox that needs each message routed to the right queue. There is a photo-upload flow that should reject pictures with no product in them. And there is a genuinely open-ended one: a knowledge assistant that answers staff questions by reasoning over a pile of internal documents.
The instinct is to reach for the same generative model for all five, because it can plausibly do all five. Ask it to validate the postcode, ask it to read the invoice, ask it to route the ticket, ask it to describe the photo, ask it to answer the question. One integration, one mental model, one bill. The bill is the first thing that gives the team pause: five features all making per-token calls to a frontier model, several of them on inputs that arrive thousands of times a day. The second is a near-miss in testing, where the postcode validator returned a pass for a malformed code.
The question underneath all five is the same. A generative model can do the task, but is it the right tool, or is there something cheaper, faster, and more predictable that fits the shape of the work better?
What actually matters
A generative LLM is not a general upgrade over every other kind of software. It is a particular tool with a particular profile: extraordinarily flexible on open-ended language, and in exchange nondeterministic, priced per token, slow relative to a lookup, prone to returning wrong answers in fluent, plausible form, and expensive to evaluate because there is rarely a single correct output to diff against. Every one of those drawbacks is acceptable when you need the flexibility. None of them is acceptable when a narrower tool would settle the task outright.
The dividing line that decides the most is whether the task is well-defined or open-ended. A well-defined task has a knowable correct answer and a describable rule for reaching it: is this string a valid postcode, does this transaction exceed a threshold, which of six queues does this ticket belong in. Tasks like that have a right answer you can test against, and the closer you get to a crisp specification the more a deterministic rule, a lookup table, or a trained classifier will beat a generative model on cost, speed, and reliability at once. An open-ended task has no single correct output: summarise this contract, draft a reply in this tone, answer this question from these documents. That is where a generative model is the right tool, because the space of acceptable outputs is too large and too fuzzy for anything narrower to cover.
The second axis is tolerance for error, and specifically the shape of the errors. A deterministic rule fails predictably: it is wrong in exactly the cases the rule doesn’t cover, and you can enumerate them. A generative model fails unpredictably and often invisibly, producing a fluent answer that is simply untrue, which is the failure mode people mean by hallucination. If a wrong answer is easy to catch and easy to fix, the unpredictability is tolerable. If a wrong answer flows straight into a downstream system, a payment, or a compliance record, the work of catching it, through evaluation, guardrails, and human review, lands back on you, and that work is part of what running the model involves.
The third is the operational profile: latency, throughput, and unit cost. A regex or a hash lookup answers in microseconds and adds nothing to the bill. A purpose-built AWS AI service answers a single-page synchronous request quickly and charges a fixed, published rate per unit: Textract expense analysis is USD$0.01 a page for the first million pages a month, Rekognition label detection USD$0.001 an image. A frontier model on Bedrock charges per input token and per output token, and generates its answer a token at a time, so latency grows with the length of the response. Volume and input length turn a rounding-error difference per call into the line item that reshapes the budget.
The fourth is validation and change control. A rule is readable, reviewable, and testable: you can prove what it does. A classic classifier has a measurable accuracy on a labelled test set. A generative feature has to be evaluated on a corpus of examples with a scoring method you build yourself, re-run whenever the prompt or the model version changes, and defended with guardrails against the outputs you never want. That evaluation and guardrail work is real engineering effort, and it comes with the model’s flexibility.
Put together, these say something simple: reach for a generative model when the task is genuinely open-ended and the flexibility is worth the unpredictability, the token cost, and the evaluation burden. Where the task is well-defined, a narrower tool is usually cheaper, faster, and easier to trust, and the model is the wrong default.
What we’ll filter on
- Task definition, is there a knowable correct answer and a describable rule, or is the output open-ended?
- Determinism needed, does the same input have to produce the same output every time?
- Error tolerance, what does a wrong answer cost, and how easily is it caught before it does damage?
- Volume and latency, how many calls, how long are the inputs, and how fast must the answer come back?
- Validation burden, can correctness be tested cheaply, or does it need a bespoke evaluation and guardrail effort?
- Flexibility required, does the task genuinely need open-ended language understanding, or is that just the convenient framing?
The landscape
Deterministic rules and lookups. A regular expression, a validation function, a hash-map lookup, a threshold comparison. For anything with a crisp specification, a postcode format, a currency threshold, a known set of routing keywords, this is the fastest, cheapest, and most reliable option there is, and it is fully testable. The failure mode is brittleness: a rule only covers the cases you wrote it for, and messy real-world input that doesn’t fit the pattern falls through. When the specification really is knowable, that trade is almost always worth it.
Traditional search and information retrieval. Keyword search, an inverted index, Amazon OpenSearch Service, or a database query. When the job is finding the right existing document or record rather than composing a new answer, plain search is cheaper and more predictable than asking a model to reproduce or generate one. It also underpins the open-ended case: retrieval feeds the documents to a generative model in a retrieval-augmented setup rather than the answer depending on what is already in the model’s weights, so search and generation are often partners, not rivals.
Classic machine learning classifiers. A model trained on labelled examples to sort inputs into a fixed set of categories: sentiment, spam, ticket routing, fraud flags. Amazon SageMaker AI, renamed from Amazon SageMaker in December 2024, trains and hosts these, and for a well-defined classification task with training data available, a purpose-trained classifier runs on infrastructure you size rather than a per-token bill, answers faster, stays deterministic for a given model version, and is measurable against a test set. It needs labelled data and retraining as the world shifts, which is its main cost.
Purpose-built AWS AI services. Managed models for specific, common tasks, offered at a fixed per-unit price with no model to train. Amazon Comprehend for entity extraction, sentiment, and language detection over text. Amazon Textract for pulling text, forms, and tables out of scanned documents. Amazon Rekognition for object, scene, face, and moderation detection in images and video. Amazon Transcribe for speech to text, Amazon Translate for language translation. For the task each was built for, they charge a published rate per unit rather than per token, and they return structured, confidence-scored output you can threshold on. They only fit inside their designed scope; push past it and you are back to a general model.
Plain software. Sometimes the honest answer is that no model of any kind is needed: a calculation, a state machine, a database join, a bit of business logic. If the task is a computation with a known procedure, code is the tool, and reaching for AI at all is the over-engineering.
Generative large language models. A frontier model on Amazon Bedrock, invoked directly or through an agent. This is the tool for open-ended language: summarising messy text, drafting and rewriting, extracting from genuinely unstructured input that no fixed schema anticipates, holding a conversation, and reasoning over documents to answer questions there is no lookup for. It gives flexibility no narrower tool can match, alongside nondeterminism, per-token cost, latency, and the evaluation and guardrail work needed to trust the output.
Evaluation
Side by side
| Approach | Best for | Deterministic | Handles open-ended input | Per-call cost | Validation ease |
|---|---|---|---|---|---|
| Rule / lookup | Crisp, knowable specifications | ✓ | ✗ | Lowest | Fully testable |
| Traditional search | Finding existing documents or records | ✓ | Partial | Low | Testable |
| Classic ML classifier | Well-defined classification with labels | ✓ (per version) | ✗ | Low | Test-set accuracy |
| Purpose-built AWS service | The specific task it was built for | ✓ (per version) | Within scope | Fixed per unit | Confidence scores |
| Plain software | Known computations and business logic | ✓ | ✗ | Lowest | Fully testable |
| Generative LLM | Open-ended language and reasoning | ✗ | ✓ | Per token, variable | Bespoke evaluation |
Reading the table against the five features: the postcode check is a rule, the invoice totals are Textract, the ticket routing is Comprehend or a classic classifier, the photo check is Rekognition, and only the knowledge assistant is genuinely a generative model. Four of the five never needed an LLM at all.
Where each task lands
A task only reaches the generative box after a knowable-answer test and a purpose-built-fit test have both failed to place it somewhere narrower. Defaulting to the model inverts that order, starting at the least predictable option and never asking whether something simpler would have done.
The solution
The postcode validator is the plain-rule case, and it is the one where using a model is actively worse. A postcode has a knowable format; a regular expression or a validation library answers in microseconds, deterministically, with a test suite that proves exactly which inputs pass. Handing that to a generative model adds nondeterminism on a task that must never be nondeterministic, charges per token for an answer local code produces at no marginal cost, and introduces the failure the team already saw in testing, a pass returned for an invalid code. When the specification is this crisp, the model is the wrong tool.
The invoice extraction is the purpose-built-service case. Pulling totals, dates, and line items out of scanned documents is exactly what Amazon Textract’s AnalyzeExpense operation exists for: it maps the varied wording on invoices onto standard field types, TOTAL, SUBTOTAL, INVOICE_RECEIPT_DATE and the rest, and returns each with a confidence score at a fixed published per-page rate. Where the layout is truly chaotic and no structured extractor copes, a generative model becomes a reasonable fallback, but that is the exception to reach for after Textract, not the default to start from.
The ticket routing is the classifier case. Sorting each message into one of six known queues is a well-defined classification problem, and if there is labelled history, either Amazon Comprehend’s custom classification or a classic model trained on SageMaker AI will route it more predictably than an LLM, with an accuracy number you can measure and watch. Their cost shapes differ too: a Comprehend real-time endpoint is charged per inference-unit-second for as long as it exists, so the bill tracks provisioned throughput rather than message length. A generative model can classify as well, but accepting nondeterminism to do a job a purpose-trained classifier does better is the pattern this whole exercise is meant to catch.
The photo check is the vision-service case. Deciding whether an uploaded image actually contains a product is object and scene detection, which Amazon Rekognition’s DetectLabels operation does at a published per-image rate, returning a confidence score for every label and accepting a MinConfidence threshold, 55 percent by default, that you can tune. It is narrower and cheaper than a multimodal LLM and returns exactly the structured signal the flow needs.
The knowledge assistant is the one genuine generative case, and it is worth seeing why it clears the bar the others didn’t. There is no fixed set of answers, no rule that maps a question to a response, and no lookup that composes a coherent explanation from several documents at once. The output is open-ended language, and the task genuinely needs reasoning over unstructured text. That justifies the token cost, the nondeterminism, and the evaluation and guardrail work, because nothing narrower can do it. Notice the shape of the good decision: the model belongs not because it can do the task but because everything cheaper cannot.
Worked example
The task is pulling the total from a stream of scanned supplier invoices, roughly forty thousand a month, feeding an accounts-payable system that pays against the figure.
Before. Each invoice page is sent as an image to a multimodal model on Bedrock with a prompt: read this invoice and return the total as a number. It works most of the time. It also, on a handful of awkward scans a week, returns the subtotal instead of the grand total, or transposes two digits, or misreads a faint figure. Because the answer is a fluent number with no confidence attached, nothing downstream flags it; the wrong total flows straight into a payment. On top of that, the bill moves with image size and response length rather than with page count, and every prompt tweak means re-checking the whole thing by hand because there is no test set, only spot checks.
After. Textract’s AnalyzeExpense operation is built for exactly this: it returns the standard field TOTAL, along with SUBTOTAL, dates and line items, each with a confidence score, at a published per-page rate that does not vary with how the page is laid out. The confidence score is what changes the risk profile. Anything above the threshold flows straight through; anything below is routed to a person before payment, so the awkward scans that used to become silent wrong payments now become a small review queue instead. The bill becomes a page count, the output is structured and thresholdable rather than a bare number, and correctness is measurable against a labelled set of invoices rather than eyeballed. The generative model was doing an open-ended reading job on a task that was never open-ended; the total was always sitting in a known field that a purpose-built extractor reads directly.
What’s worth remembering
- A generative model is the most flexible tool available and the least predictable and hardest to validate, billed per input and output token rather than per unit; match the tool to the task instead of defaulting to the model.
- Well-defined tasks with a knowable correct answer belong to rules, lookups, classifiers, or purpose-built services, which are faster, deterministic, priced per unit, and testable.
- A generative LLM is the right tool on genuinely open-ended work: summarisation, drafting, extraction from unanticipated unstructured text, conversation, and reasoning over documents.
- Reach for the model because nothing narrower can do the task, not because the model can also do it; being capable is not the same as being the right fit.
- Deterministic tools fail predictably in cases you can enumerate; generative models fail invisibly with fluent wrong answers, so weigh what a wrong answer costs and how easily it is caught.