Exam-style
A media company's Bedrock summarisation service runs on a current-generation foundation model, handles roughly 2 million tokens an hour, all day and every day, and the product team has committed to a p99 latency target. The per-token bill keeps climbing. What should they move to?
Reveal the answer
E. Reserve input and output tokens per minute on the Reserved tier, sized to the sustained rate
Steady, high, round-the-clock volume with a latency commitment is what the Reserved tier is priced for: input and output tokens per minute are reserved separately at a fixed price per 1K TPM, billed monthly on a one- or three-month duration, and traffic above the reservation overflows to the Standard tier rather than throttling, so the p99 commitment survives a spike. Provisioned Throughput is the older Model Unit reservation, and its supported base models stop generations back, which leaves it as the serving path for custom fine-tuned models rather than a capacity lever for a current-generation one. Backoff manages throttling but does nothing for the latency target or the bill. Savings Plans cover compute services and do not apply to Bedrock model inference. Batch inference is cheaper but asynchronous with a 24-hour turnaround, so it cannot serve a p99 target.
Q. Steady high-volume Bedrock traffic with latency guarantees. Reserve capacity or stay on-demand?
A. Reserve it. On a current-generation base model the reservation to buy is the Reserved tier. You pick input and output tokens per minute, pay a fixed price per 1K TPM on a one- or three-month duration, and get predictable latency, with traffic above the reservation overflowing to the Standard tier. Provisioned Throughput reserves Model unitThe billing block Provisioned Throughput is sold in – one unit delivers a fixed tokens-per-minute rate for a specific model. instead, and its supported base models stop generations back. That leaves it as the serving path for a customised model whose base offers no on-demand custom serving. On-demand suits spiky or low volume, paying per token with no commitment.
Why? Match the pricing model to the traffic shape: steady-and-high favours a reservation, and which reservation you can buy depends on whether the model is current-generation or custom.