How to Cut a Bedrock Bill Without Hurting Quality

The situation

Bedrock spend for the company’s AI platform was USD$18k last quarter. It’s tracking toward USD$46k this quarter, and the graph keeps curving upward. No single service is responsible; the support assistant is up 40%, the ticket classifier is up 120%, the marketing-copy drafter is up 80%, and a new internal-search tool the research team launched last month is already second on the leaderboard.

Finance wants a plan with a target: get next quarter inside USD$35k without hurting user-facing quality. Product wants reassurance that the features they’ve scoped for next quarter can still ship. The platform team, which is where the bill lands, wants tools they can apply repeatedly rather than a one-time cost-cut exercise.

Token accounting has been on for a while. The breakdown reads:

• Input tokens: 68% of spend. Long retrieval contexts, uncompressed system prompts, verbose few-shot examples.
• Output tokens: 28% of spend. Chatty default response styles, unstructured output the model expands on.
• EmbeddingEmbeddingA fixed-length vector of floats that represents a piece of text (or image, or other thing) in a space where similar meanings sit close together. calls: 3%.
• Other (evaluations, fine-tuningFine-tuningContinuing to train an already-trained model on a smaller dataset to adapt its behaviour. runs): 1%.

Everything runs on demand, on the Standard service tier. There is no Provisioned Throughput and no reserved capacity. The model mix is 70% Claude Sonnet 5 (anthropic.claude-sonnet-5), 20% Nova Pro, 5% Claude Haiku 4.5, and 5% others.

What actually matters

Input tokens carry more than two-thirds of this bill, so anything that shortens what goes in returns more than twice what the same effort returns on the way out. That ordering holds for most retrieval-heavy platforms, and it is worth confirming from the account’s own numbers before choosing a lever, because a portfolio of long-form generators inverts it.

The levers differ sharply in what they can damage. Routing a service to a smaller model, and trimming what the retriever returns, both change the text the model sees, so both can change answers and both need an evaluation before they ship. Caching, capacity commitments and output caps do not change the evidence or the instructions, so the risk there is operational rather than editorial. Reversibility splits the same way: a request flag or a prompt version can be rolled back this afternoon, a term commitment cannot.

Eligibility is set per model, and this is where cost plans usually break. Explicit prompt caching has a minimum prefix length that varies: Claude Sonnet 5 creates a cache checkpoint only once 1,024 tokens have accumulated, and Claude Haiku 4.5 not until 4,096. Provisioned Throughput covers a published list of models that current Claude releases are not on. Service tiers vary too: Claude Haiku 4.5 offers a Reserved tier, Claude Sonnet 5 offers only Standard. A lever that saves money on one model returns nothing on the model next to it, and moving a service between models can switch a lever off.

All of it depends on seeing what is happening. The bill is an aggregate, the savings are per call, and the changes are per service. Without per-service and per-model attribution, every change is a hypothesis with no way to check it.

What we’ll filter on

1. Savings magnitude, how many percent off the bill, realistically?
2. Quality risk, does this change the text users see?
3. Implementation effort, hours, days, or weeks of engineering?
4. Blast radius, how many services does this touch?
5. Reversibility, can we roll this back if it goes wrong?

The landscape

Model right-sizing. Take the top five services by spend, evaluate each against a smaller model in the same family, switch the ones that don’t regress. Claude Haiku 4.5 (anthropic.claude-haiku-4-5-20251001-v1:0) and Nova Micro are the usual destinations. Bedrock does have a managed version of this, intelligent prompt routing, which predicts response quality per request and routes within one model family; it is in preview and its supported list covers Claude 3 and 3.5, Nova Lite and Pro, and Llama, so routing across current Claude releases stays application-level for now. Typical savings when half the calls can move down: 30-50% of the affected service’s bill. Reversible by pointing the application at the previous Prompt Management version.

Prompt caching on stable prefixes. Implicit prompt caching needs no request changes and reuses eligible prefixes on a best-effort basis. Explicit prompt caching adds cache checkpoints, up to four per request on Claude models, in the system, messages and tools fields. Cache reads bill at the model’s cache-read rate, roughly a tenth of the input rate where AWS publishes both; cache writes bill above the standard input rate, 1.25× on the models that publish that figure. The default TTL is five minutes and resets on every hit, and setting "ttl": "1h" extends it to an hour on Sonnet 5 and Haiku 4.5. Caching applies to on-demand endpoints only, not to batch inference. Typical savings on input costs: 30-50% when the cached prefix is a large fraction of the prompt.

Provisioned Throughput. Commit to a number of model units per model with no commitment, a 1-month term, or a 6-month term, billed hourly whether the capacity is used or not. The published model list runs to Nova Micro, Lite, Pro and Nova 2 Lite, Titan, and older Claude and Llama releases; current Claude models are not on it, and inference profiles don’t support Provisioned Throughput at all. Typical savings: 40-60% on the committed portion, and nothing if usage doesn’t fill the commitment. A term commitment can’t be deleted before it ends.

Reserved service tier. The committed-capacity route for models that offer it, including Claude Haiku 4.5. Input and output tokens-per-minute are reserved separately for a 1-month or 3-month term at a fixed price per 1,000 TPM, billed monthly, with a floor of 100,000 input TPM and 10,000 output TPM. Traffic above the reservation overflows to Standard rather than failing. Access goes through the AWS account team. Claude Sonnet 5 offers Standard only, so this does nothing for 70% of the current mix.

Output-length constraints. maxTokens in the request, plus system-prompt instructions that keep responses short. “Respond in at most 75 words, no preamble” cuts output tokens on summary-style tasks by 30-50%. Bedrock’s structured outputs feature constrains responses to a schema; Claude Haiku 4.5 supports it on the bedrock-runtime endpoint and Claude Sonnet 5 does not, so a service that wants schema-constrained JSON has a model constraint attached. Effort: prompt edits. Reversible.

Retrieval tightening. numberOfResults on a Knowledge Base query accepts 1 to 100, and most retrievers are set well above what the generator uses. Drop top-k, cut chunk size, or add a rerankingConfiguration so a wide candidate set is narrowed to a few high-scoring chunks before generation. Typical savings on retrieval-heavy services: 20-40% on input. More aggressive trimming can hurt recall, so evaluate before shipping.

Response caching for repeated queries. A hash of (prompt, model, params) to a cached response in ElastiCache or DynamoDB with a TTL, so a cache hit skips the model call. Works for FAQ-style traffic where the same question arrives hundreds of times; works poorly for conversational traffic with long session context. Typical savings: up to 100% on cacheable calls, entirely dependent on traffic shape.

Batch inference. Prompts go to S3, responses come back asynchronously, and AWS lists batch pricing at half the on-demand rate. It rules out tool calling, structured outputs and prompt caching, and it can’t run against a provisioned model, so it suits offline scoring and bulk classification rather than anything a user is waiting on.

Global inference profiles. A geographic profile keeps requests inside a geography; a global profile routes worldwide and AWS puts the saving at approximately 10%. Either way the price is calculated from the Region the profile is called from, so moving the caller is what changes the rate, not where the request lands. The rest of the saving here comes from consolidating duplicated deployments into one account with per-service attribution.

Evaluation

Side by side

Stacking is not additive, since several of these compete for the same tokens. A realistic quarterly programme combines three or four of the low-quality-risk levers and lands 30-50% down without retraining a model or changing a product feature.

Measuring before and after

Every lever here is a form of token efficiency, and the practice underneath all of them is estimation and tracking. Estimate before building: run a representative prompt through the CountTokens API where the model supports it, or fall back to the rough four-characters-per-token rule for a first pass, and a feature’s cost per call is known before it ships. Track once it’s live: InputTokenCount and CacheWriteInputTokens are both CloudWatch metrics, and CloudWatch also carries ModelId, ServiceTier and ResolvedServiceTier dimensions, so a prompt that has grown from 2,000 tokens to 8,000 shows up as a trend on a graph instead of a jump on an invoice. Context-window planning is the same arithmetic aimed at the window: Claude Sonnet 5 takes 1M tokens of context and returns at most 128K, Claude Haiku 4.5 takes 200K and returns at most 64K, so reserve the output allowance first and let the remainder set the input budget.

The three levers that move those numbers have names worth using. Prompt compression is the rewrite, the same instruction coverage in half the tokens. Context pruning is what happens to retrieved material before it reaches the model: boilerplate, navigation headers, citation markers and low-scoring chunks come out of the context so the generator sees the evidence and little else. Response limiting is the output side. maxTokens is the hard control, since generation stops at the cap regardless of what the model was producing; a prompt instruction (“at most 75 words, no preamble”) is the soft one, and the model follows it most of the time. stopSequences is the third, ending generation at a string nominated up front.

The solution

Model routing. The largest lever, and the one that needs the most care. Start with the ticket classifier, a seven-class job with clear rubrics, and evaluate Nova Micro against the current Nova Pro baseline on a 500-ticket sample; if the decision rule holds, switch. Repeat for the marketing-copy drafter’s first-pass generation, with revision staying on Sonnet 5, the internal-search summariser, and the intent-detection step at the front of the support assistant. Those three move to Claude Haiku 4.5. Keep Sonnet 5 for the conversation turns where fluency matters. Expected saving: about 25% of the overall bill, over two to three weeks.

Prompt caching, with the minimums checked first. The support assistant’s 1,800-token system prompt clears Sonnet 5’s 1,024-token floor and caches. The classifier’s 600-token rubric does not: a checkpoint below the minimum still returns a successful inference, it just never caches, so that prompt bills as ordinary input and the team should stop expecting otherwise. The intent-detection step is the awkward case, because moving it to Haiku 4.5 raises its floor to 4,096 tokens and switches its caching off, which is a trade the routing saving comfortably covers. Where caching does apply, watch cacheReadInputTokens and cacheWriteInputTokens in the response rather than assuming a hit. Expected saving: about 12% of the overall bill, within a week.

Output-length constraints. System-prompt lines and maxTokens caps. The summariser gets “respond in at most 75 words, no preamble”; the classifier moves to a fixed JSON schema; the marketing-copy drafter keeps its longer outputs but gains “no meta-commentary, no recap, no follow-up suggestions”. Expected saving: about 5% of the overall bill.

Retrieval tightening. The support assistant and internal search both over-retrieve. Move numberOfResults from 10 to 5 and add a reranker over a 20-chunk candidate set. Chunk size goes from 500 tokens to 300, which needs a re-index and also improves precision. Ship if the retrieval evaluation doesn’t regress. Expected saving: about 7%.

Committed capacity. Skip it, and leave it out of the projection. Provisioned Throughput doesn’t cover the current Claude models, and the Reserved tier that does cover Haiku 4.5 needs 100,000 input TPM reserved before it starts, which no single service here comes close to sustaining. Revisit once the daily summariser’s traffic has grown.

Batch inference and response caching. The nightly bulk re-classification of the ticket backlog moves to batch at half the on-demand rate, since nothing there needs tool calling or a schema. Response caching goes on a small FAQ endpoint the marketing team uses, and stays off the support assistant, which is conversational and rarely repeats a prompt exactly.

Worked example

A quarter of classifier traffic: 5,000,000 calls, 600 input tokens each (rubric plus ticket), 20 output tokens each. That is 3,000M input tokens and 100M output tokens. On Nova Pro, at the published US East on-demand rate of USD$0.80 per million input and USD$3.20 per million output:

Input:  3,000M × USD$0.80/M = USD$2,400
Output:   100M × USD$3.20/M =   USD$320
Total: USD$2,720

On Nova Micro, at USD$0.035 per million input and USD$0.14 per million output:

Input:  3,000M × USD$0.035/M = USD$105
Output:   100M × USD$0.14/M  =  USD$14
Total: USD$119

That is about 96% off one service, USD$2,601 saved for the quarter. The evaluation came first, with a decision rule of F1 within 0.03 across all categories: Nova Micro scored 0.93 against Nova Pro’s 0.95 on a 500-example set, inside tolerance. The application then points at a new Prompt Management version, and rollback is repointing it at the old one. CloudWatch confirms the token counts fall and the quality signal holds. Total engineering time: roughly two days for eval design, eval run, and rollout.

The classifier is a small line on the overall bill, and the same method applied to three larger services is the USD$12k in the waterfall.

What’s worth remembering

1. Cost control is a stack of levers. Three or four combined produce more than any one of them, and they are not additive because several compete for the same tokens.
2. Model right-sizing is the largest lever for most portfolios, because the default drifts up to the best available model. Evaluate, then move down.
3. Prompt caching turns on in hours, but the minimum prefix length varies by model, and a checkpoint below it caches nothing while the call still succeeds.
4. Committed capacity follows the model card: Provisioned Throughput lists the models it covers, and the Reserved tier has a tokens-per-minute floor that small services never reach.
5. Retrieval over-fetch is a quiet cost. Top-k of 10 where 3 would do sends seven surplus chunks per call, and a reranker narrows a wide candidate set before generation.
6. Per-service, per-model attribution comes before all of it. Tag every invocation and dimension every metric, or the changes are untestable.