How to Match Bedrock Pricing to Workload Rhythm

The situation

Two production services on Bedrock, both hitting Claude Sonnet 5, both starting to feel the limits of on-demand.

Service A: the customer-facing assistant. Runs 24/7. Peak traffic is 8 requests per second (US and EU business hours overlapping); trough is around 2 requests per second (overnight in both regions). Median request consumes 1,500 input tokens and produces 200 output tokens. Runs ~10 million requests per month. Latency matters, product has a p95 SLA of 2 seconds end-to-end; Bedrock latency is most of that budget. The service has started hitting on-demand throttling during peak, seeing occasional ThrottlingException errors that the retry logic masks but that add latency spikes.

Service B: the weekly report generator. Runs for about 6 hours every Sunday morning. Generates ~80,000 reports in that window, each consuming ~3,000 input tokens and producing ~800 output tokens. Rest of the week, zero traffic. Latency per request doesn’t matter, reports aren’t interactive, but the job has to finish within the 6-hour window because downstream distribution kicks off on Sunday afternoon.

Both services are candidates for something better-fitted than plain on-demand. The question is what shape of pricing fits each, and what, if anything, to commit.

What actually matters

Inference pricing for managed foundation models tends to come in two shapes.

Pay-as-you-go is per-tokenTokenThe unit of text an LLM actually sees – usually a short character sequence, not a whole word. : input tokens at one rate, output tokens at another (typically several times higher). No upfront commitment; pay exactly what’s used. Subject to account-level throttling quotas, requests-per-minute and tokens-per-minute, per modelModelA trained set of weights plus the architecture that makes them useful – the thing you load up and run inference against. , which can be raised by support request but cap the burst capacity. Within pay-as-you-go there is room to pay a premium for faster handling or take a discount for patient handling, without any reservation.

Committed capacity reserves a slab of throughput on a model for a fixed term: a guaranteed tokens-per-minute number, billed flat regardless of utilisation. Latency is more predictable because the capacity is reserved rather than shared, and traffic above the reservation spills back to pay-as-you-go rather than failing.

The core trade: money for predictability. Committed capacity buys guaranteed throughput at a fixed price; pay-as-you-go charges only for what’s used but can throttle and has variable latency.

The first decision is whether a commitment is economically justified at all. For a workload running 24/7 at meaningful volume, the total token spend is large enough that a well-sized commitment can undercut pay-as-you-go. For a workload that runs a few hours a week, paying for a month of capacity to serve those hours is wasteful no matter how favourable the rate.

The second is how to size the commitment. The commit has to cover peak throughput, not average, or it has to be deliberately sized below peak with a plan for spillover. Over-committing wastes money on idle capacity; under-committing means peak traffic spills back to pay-as-you-go (permitted, at the pay-as-you-go rate).

The third is how latency behaves on the two modes. Pay-as-you-go latency is driven by shared-tenancy inferenceInferenceRunning a trained model to produce output – as opposed to training it. queueing; during peaks, requests wait in queue. Reserved capacity keeps queue depth low because nobody else is using it. The observable shape: p50 latency similar; p95 and p99 materially better on committed capacity during peak hours.

The fourth is commitment length. Shorter terms come at a higher monthly rate; longer terms come with a discount and a longer lock-in. The choice mirrors any other reserved-capacity calculus: cheaper per-month buys less flexibility.

The fifth is whether routing tricks can postpone the decision. Cross-region inference routing, a single call dispatched to whichever region has capacity, can absorb bursts on pay-as-you-go without a commitment. It works with pay-as-you-go; it’s less relevant once capacity is reserved.

It’s also worth asking what happens if traffic doubles or halves. A commitment is rigid for its term. If traffic doubles next quarter, the team needs a bigger reservation (or spillover at pay-as-you-go rates). If it halves, the bill stays the same. Committed capacity suits workloads with stable, predictable envelopes, not workloads in the middle of a growth or shrinkage curve.

What we’ll filter on

1. Latency predictability, p50, p95, p99 under peak load?
2. Monthly cost at expected usage, which pricing shape wins at this traffic profile?
3. Cost at worst-case traffic, what happens when actual usage deviates from plan?
4. Commitment flexibility, scaling up, down, or out mid-term?
5. Operational overhead, what changes in the day-to-day with each shape?

The pricing-shape landscape

Bedrock’s runtime API takes an optional service_tier parameter (reserved, priority, default, or flex), and most of the landscape below is that one parameter; only the Reserved tier and the legacy option at the end involve a purchase.

1. Standard tier (the default). Baseline on-demand. No commitment, pay per token, subject to account throttling limits. Variable latency; spiky. Suits unpredictable workloads, low volumes, bursty occasional jobs. Requests without a service_tier land here.

2. Standard + throttle increase. Request a higher RPM/TPM limit via support. Buys more headroom on on-demand; doesn’t change latency characteristics. Free (you don’t pay more per token), just asks AWS for a bigger queue. The on-demand quota is shared across the Standard, Priority, and Flex tiers.

3. Standard + cross-region inference. Configure an inference profile that routes across multiple regions. Increases effective throughput at the cost of slightly higher cross-region latency. Cross-region inference profiles apply to on-demand traffic (no reservation needed); very useful for bursty workloads.

4. Flex tier. service_tier: "flex" takes a pricing discount in exchange for tolerating longer processing times. Same per-token shape, same shared quota, no commitment; requests sit behind Standard and Priority when capacity is tight. Fits model evaluations, summarisation sweeps, and agentic background work that is online but not urgent.

5. Priority tier. service_tier: "priority" pays a premium over Standard for the fastest response times, prioritised ahead of Standard and Flex, with no reservation and no commitment. Fits customer-facing flows whose latency pain is real but whose volume doesn’t justify reserving capacity around the clock.

6. Reserved tier. A capacity reservation: pick input and output tokens-per-minute numbers separately (minimums 100K input TPM, 10K output TPM), pay a fixed price per 1K TPM, billed monthly, on a 1-month or 3-month duration, arranged through the AWS account team. Targets 99.5% uptime for model response, and when traffic exceeds the reservation it overflows to the Standard tier automatically, so the spillover plan is built in. One sizing gotcha: prompt-cache writes (CacheWriteInputTokens) count toward the input reservation alongside InputTokenCount.

7. Batch Inference API. A separate pricing tier for batch workloads. Submit a manifest of requests, Bedrock processes them within 24 hours at roughly 50% of the on-demand rate. Not subject to real-time throttling. Perfect for Service B-style batch jobs.

8. Provisioned Throughput, the legacy corner. The older reservation construct, bought in Model Units per month. Still real, but its supported-model list stops generations back (on the Anthropic side nothing newer than Claude 3.5 Sonnet v2), and its remaining everyday job is serving custom fine-tuned Llama and Titan models, which require it. For a current-generation foundation model, the Reserved tier is the equivalent lever; treat PT as the answer to a legacy or custom-model question, not a current capacity-planning one.

Side by side

Service A and Service B, placed

The picks in depth

Service A: Reserved tier, 1-month duration. The traffic shape, steady daily pattern, stable week to week, fits a monthly reservation. The reservation is sized in tokens-per-minute, input and output separately. Peak works out to roughly 720K input TPM and 96K output TPM (8 rps × 1,500 in / 200 out), comfortably above the tier’s minimums. Reserve roughly 75% of that, about 540K input and 72K output TPM, not 100%: covering the full peak would pay for capacity that sits idle two-thirds of the day, and traffic above the reservation overflows to the Standard tier automatically at Standard rates, which are the rates we were happy paying for those hours anyway. One sizing detail that bites teams using prompt caching: cache writes count toward the input reservation, so size from InputTokenCount plus CacheWriteInputTokens in CloudWatch, not from input tokens alone.

Rough math: at a fixed price per 1K reserved TPM (figures vary by model and change regularly), the monthly reservation sits in the low-to-mid five figures; on-demand for the same 75% of traffic was in the mid five figures. Savings: meaningful, 30-40% depending on the exact rates at commit time.

Side effect: p95 latency drops because the reserved capacity removes shared-tenancy queueing during peaks, and the tier targets 99.5% uptime for model response. Product sees the SLA compliance rate improve from 94% to 99%. If the latency pain had come without the throttling, the lighter fix would have been service_tier: "priority" on the hot paths: faster handling at a per-request premium, no reservation, no commitment.

Service B: Batch Inference API. A reservation would be wildly wasteful here, 6 active hours per 168-hour week. The Batch Inference API is exactly the correct tool: submit a manifest of requests, Bedrock processes them at roughly 50% the on-demand rate within 24 hours. The Sunday-morning window starts earlier and accepts the batch API’s less-than-24-hour turnaround; downstream distribution kicks off Sunday afternoon. The Flex tier is the near-miss here: it discounts patient online traffic, but this job isn’t online at all, it’s a manifest, and batch discounts deeper.

Rough math: 80,000 reports × 3,800 tokens average = 304M tokens per Sunday. At half the Standard rate, the weekly bill drops by roughly half. No ops overhead beyond the batch submission code. Zero commitment risk, a week when no reports run costs zero.

What we keep where. Ad-hoc queries and experimentation notebooks stay on Standard, the tier designed for exactly that. Evaluation jobs and other patient background inference move to service_tier: "flex" for the discount, a one-line change per call.

Rollout. Service A moves to the Reserved tier through the account team over two weeks: week one, reserve half the target and monitor utilisation and Standard-tier overflow in CloudWatch (the ServiceTier and ResolvedServiceTier dimensions show which tier actually served each request); week two, true up to the full reservation once the math is confirmed. Service B migrates to the Batch Inference API in a sprint, the submit/poll code is straightforward; the existing real-time invocation loop replaces with a batch-job state machine.

A worked example: the bill before and after

Current monthly spend on both services, all Standard on-demand:

Service A (24/7 assistant):
  15B input + 2B output tokens/month at Sonnet ($3/M in, $15/M out)
  ≈ $75,000/month on-demand

Service B (weekly batch):
  ~300M tokens/week × 4 weeks ≈ 1.2B tokens/month at Sonnet
  ≈ $6,700/month on-demand

After the changes:

Service A:
  Reserved: 540K input + 72K output TPM at $N/1K TPM/month  ≈ $33,000
  Standard-tier overflow (peaks above the reservation)      ≈ $19,000
  Subtotal                                                    $52,000

Service B:
  Batch Inference API (50% of Standard)                     ≈ $3,350

Combined monthly spend drops from roughly $81,700 to $55,350, a saving of about a third, latency improvement on Service A as a bonus, and two workloads better-matched to the pricing model that fits them.

What’s worth remembering

1. Standard on-demand prices predictability at zero; the Reserved tier prices it per month. The question is whether you need predictability enough to pay for it.
2. The tiers are mostly one API parameter: service_tier set to priority, default, or flex trades speed against price per request with no commitment; only Reserved involves a purchase.
3. Reserved fits steady workloads with stable envelopes: predictable daily traffic, month-over-month similarity, no imminent order-of-magnitude changes. Terms are 1 or 3 months, arranged through the account team.
4. Reserve 70-80% of peak, not 100%. Overflow to the Standard tier is automatic and bills at Standard rates, which you were paying before; reservations are sized in input and output TPM separately, and prompt-cache writes count toward the input side.
5. The Batch Inference API is the correct tool for batch: ~50% of Standard, no commitment, up to 24-hour turnaround. Flex is for patient online traffic; batch is for manifests.
6. Priority buys the fastest handling at a premium with no reservation, the right lever when the pain is latency rather than throughput.
7. Reserved-tier latency benefit is real during peaks: p95 and p99 improve, p50 stays roughly the same, and the tier targets 99.5% uptime. Measure the shape before committing.
8. Cross-region inference is a free capacity lever on on-demand. Configure an inference profile, route to multiple regions, multiply effective throughput.
9. Throttle-limit increases are free headroom, and the on-demand quota is shared across Standard, Priority, and Flex.
10. Provisioned Throughput is the legacy reservation: still how custom Llama and Titan models are served, still sold for older foundation models, but for current-generation models the Reserved tier is the construct to reach for.

Two workloads, two pricing shapes, one bill that dropped by a third, and a latency SLA that product stopped complaining about. The lever wasn’t a cheaper model; it was matching each workload to the pricing shape that fits its rhythm.