Exam-style
A support-triage application on Bedrock costs AUD$40,000 a month. Most calls are simple classification of a short ticket, they all go to the newest flagship model, and every request repeats the same 6,000-token policy document as context. Answers are needed while the agent is on the call, and quality must not drop. What are the first levers?
Reveal the answer
E. Route the classification calls to a smaller model and cache the repeated policy context
Bedrock cost is model choice multiplied by tokens, so sending simple classification to a smaller model and caching the 6,000-token prefix reduces both terms at once. Check the cache checkpoint minimum for whichever model you land on. It runs from 512 tokens on Claude Opus 5 to 1,024 on Claude Sonnet 5 and 4,096 on Claude Haiku 4.5. A request whose prefix falls under that minimum still succeeds, without caching. Cache reads bill at the model’s cache-read rate, about a tenth of the input rate on Claude models, while the write that fills the cache bills above it. Provisioned Throughput bills hourly per model unit on a no-commitment, one-month or six-month term, and its base-model list stops at Claude 3.5 Sonnet v2, so it cannot be bought for the flagship this workload runs on. Customising a model is worse, because a customised model can only be served through Provisioned Throughput, so the hourly bill runs whether tickets arrive or not. Batch inference costs half as much, but it writes results to S3 asynchronously and does not support prompt caching, so a live queue cannot use it. Guardrails apply content, denied-topic, word, sensitive-information, grounding and automated-reasoning policies. None of them acts on input length, and each evaluation bills per 1,000 text units on top of inference.
Q. Your Bedrock bill is high but quality must hold. First levers?
A. Right-size the model per task, a smaller model wherever one suffices, and cache the context that repeats on every call. Prompt cachingReusing the model’s already-processed prefix (system instructions, fixed context) across calls so you don’t pay to re-read it every time. reuses an already-processed prefix, so check the checkpoint minimum for that model first: 512 tokens on Claude Opus 5, 4,096 on Claude Haiku 4.5. Then trim prompt and output tokens. Where latency is elastic, the Flex service tier and Batch inferenceSubmitting a bulk job of model calls to run asynchronously at a lower per-token price, trading immediacy for cost. both run at half the Standard price, Flex per request and batch asynchronously through S3, and the model card lists the tiers available for a given model. Provisioned ThroughputReserved Bedrock capacity bought by the hour for a fixed term, paid for whether traffic fills it or not. and the Reserved tier are reservations for steady high volume, and which reservation you can take out is set per model.
Why? Cost is model choice multiplied by tokens. Match model strength to the task, then stop re-sending what has not changed.