The situation
A logistics company has three model-backed features converging on the same release train. The customer-facing assistant calls a hosted model that AWS operates; its traffic is spiky and daytime-shaped, near zero overnight. A document classifier is midway through a training run on the team’s own labelled data, due in about three weeks. And the research group has a specialised extraction model they trained on their own hardware, with the weights sitting in an S3 bucket waiting for somebody to decide what happens next.
Finance has asked for a twelve-month serving forecast covering all three. It is a reasonable request and nobody can answer it. The three features do not merely cost different amounts; they are metered on different things. One accrues charges only when a request arrives. One will accrue them by the hour whether or not anybody uses it. The third has not been decided yet, and the team has been assuming that decision is theirs to make on price.
That last assumption is the one that hurts. Two of the three have already had their billing shape settled by choices made weeks ago, before anybody drew up a forecast, and reversing either means retraining rather than reconfiguring.
What actually matters
Start with where the options come from, because that is the part teams expect to control and mostly cannot. The set of ways a model can be served is fixed by its provenance: whether the weights are ones the platform hosts for everybody, ones you adapted through the platform’s own training, or ones you produced elsewhere and brought with you. Each of those origins opens a different set of surfaces, and within the middle one the specific model you started from narrows it further. By the time there is a bill to look at, the option set is already decided.
The second is that the billing units differ in kind rather than in rate. One surface meters the tokens a request consumes. Another meters reserved capacity by the hour, regardless of whether requests arrive. Another meters the minutes during which capacity is live. Another meters the instances you keep running. You cannot compare these by looking at their prices, because they are prices of different things. A comparison only becomes meaningful once you supply a traffic shape, and the same two surfaces will swap places depending on whether the workload is a steady grind or ninety busy minutes a day.
The third is what happens when nothing is happening. A unit that meters time keeps metering overnight, at weekends, and through the quiet fortnight after a launch. A unit that meters consumption stops. Between them sits the surface that stops charging after a period of idleness but makes the next caller wait while capacity comes back, which trades latency on the first call after a lull against a bill that stops. Whether that trade is acceptable is a question about the workload’s tolerance, not about its budget.
The fourth is commitment. Where capacity is reserved, it can usually be reserved for longer in exchange for a lower rate, which is a straightforward trade of price against flexibility and a poor bet on a workload whose volume nobody has measured yet. The discount is real and so is the lock-in, and a term chosen before the first month of production traffic is a guess, whatever the saving on paper says.
The last is that serving is not the only line on the invoice. Adapting a model costs something at training time, usually metered on the data processed and the number of passes over it. The resulting artefact then costs something every month it exists, whether or not it is serving, until somebody deletes it. Teams forecast the inference and are surprised by the rest, and the rest is what keeps accruing after a feature is switched off.
What we’ll filter on
- Where did the weights come from, and does that origin leave more than one serving surface open?
- Does the bill follow the traffic, or does it follow the clock?
- Does it stop when the workload stops, and how long does the first request after a quiet period take?
- What commitment does it ask for, and how reversible is that commitment?
- What accrues when nothing is being served: reserved capacity, idle instances, stored artefacts?
The landscape
On-demand invocation of a hosted model. Call the model, pay for the tokens the call consumed, input and output priced separately with output usually dearer. No capacity to reserve, no floor, nothing accruing between calls. This is the default surface for models AWS hosts, and it is the one every prototype starts on. Two variants sit alongside it. Batch inference runs the same prompts asynchronously from files in S3 at half the on-demand token rate, with results arriving in hours rather than seconds, and it does not support tool calling or structured output. Cross-Region inference profiles spread load across Regions to soften throttling. There is no routing charge, and the rate is the source Region’s. A global profile, which can route to any commercial Region, runs about ten percent cheaper than a geography-scoped one. Fits spiky, interactive, and exploratory workloads, which is most of them.
Provisioned Throughput. Reserve capacity in Model unitThe billing block Provisioned Throughput is sold in – one unit delivers a fixed tokens-per-minute rate for a specific model., each delivering a defined throughput for one specific model, and pay by the hour for as long as the reservation exists. Most eligible models can be reserved with no commitment at the highest rate, or for a one-month or six-month term at progressively lower ones. Billing continues until the reservation is deleted, and a committed term cannot be deleted before it ends. The eligibility rule is the part that catches people: AWS publishes a list of foundation model IDs that Provisioned Throughput can be purchased for, covering those base models and any model you customised from one through Bedrock’s own training. Nothing outside that list is eligible, whatever else you may have in the account. Fits steady high volume, a guaranteed throughput floor, and the customised models that have no other option.
On-demand serving of a customised model. Some bases, once customised through Bedrock’s training, can be deployed for on-demand inference and billed per token, with nothing reserved. The list is short and specific: Nova Micro, Nova Lite, Nova 2 Lite and Nova Pro in US East (N. Virginia), and Llama 3.3 70B Instruct in US West (Oregon), and the customisation job has to have run on or after 16 July 2025. Other bases have no such deployment, and for those Provisioned Throughput is the only path, priced on the base model’s unit rate. Which side of that line a customisation falls on is a property of the base you picked at the start, so it is a question to settle before training rather than after. The route you take to customise does not change the answer; the base does.
Custom Model Import. Bring weights trained elsewhere into Bedrock’s managed serving, in one of four Regions, and call them through the same API surface as anything else. Billing runs by the Custom Model Unit minute of active use, in five-minute windows from the first successful invocation, plus a monthly storage charge levied per unit rather than per model, so a model that needs three units stores at three times the rate. Five minutes with no invocation scales the model copies to zero, and the next call takes a cold start of tens of seconds, longer for bigger weights. Imported models are not on the Provisioned Throughput eligibility list, so this per-minute meter is the billing shape rather than one option among several, and batch inference is not available to them either. Fits sporadic or clinic-shaped traffic on weights you own, across a published set of architectures (Llama, Mistral, Mixtral, Qwen, GPT-OSS and a few more) with embedding models excluded.
Self-hosting on SageMaker. Run the weights on infrastructure you manage. A real-time endpoint bills instance-hours for as long as it exists, giving predictable latency and full control, and you pay through every quiet hour. Serverless inference bills the compute duration by the millisecond plus the data processed, and scales to zero between requests, but it tops out at 6 GB of memory with no GPU, which rules large models out rather than merely making them slow. Asynchronous inference sits between them for work that tolerates queuing. Fits architectures Bedrock does not support, deployment control Bedrock does not expose, and teams who would rather own the operational surface than the constraint list.
Evaluation
Side by side
| Surface | Applies to | Billing unit | Stops when idle | Commitment |
|---|---|---|---|---|
| On-demand | Hosted models | Per token | ✓ | None |
| Batch | Hosted models | Per token, 50% of on-demand | ✓ | None |
| Provisioned Throughput | Listed base models, and Bedrock customisations of them | Per model-unit hour | ✗ | None, 1 month, or 6 months |
| Custom model deployment | Nova Micro/Lite/2 Lite/Pro, Llama 3.3 70B | Per token | ✓ | None |
| Custom Model Import | Supported architectures trained elsewhere | Per unit-minute active | ✓ (after 5 idle min) | None |
| SageMaker real-time | Anything | Instance-hours | ✗ | None (or Savings Plans) |
| SageMaker serverless | CPU models up to 6 GB | Per compute-millisecond | ✓ (cold start after) | None |
Which surfaces a model can reach
Reading the diagram against the three features: the assistant has the widest choice and should keep it, the classifier’s choice was made when somebody picked its base, and the extraction model has one managed option and one self-managed one, with reservation available for neither.
The solution
The assistant stays on demand. Spiky daytime traffic against a hosted model is the case per-token billing exists for: nothing accrues overnight, nothing is committed, and the bill tracks the feature’s actual use closely enough that finance can forecast it from a request count. Provisioned Throughput would be a reasonable thing to revisit only if the traffic flattened into a steady, high, predictable grind, or if on-demand throttling started adding user-visible latency, which a guaranteed floor would fix. Neither is true, so the work here is measurement rather than architecture: know the token volume per request and per day, and the forecast follows.
The classifier’s serving shape is already decided and the team should find out which way before the training run finishes rather than after. If its base is one that supports on-demand custom serving, the classifier deploys and bills per token, at the same rate as base inference for the Nova bases, and it behaves like the assistant for forecasting purposes. If not, Provisioned Throughput is the only path, and the forecast changes character completely: a reserved unit bills every hour it exists, so a classifier processing a few thousand documents in a daily batch would spend most of the month paying for capacity nobody is using. That is where the unit count and the term drive the bill, and where a nightly batch window rather than a live endpoint may be the cheaper design. Either way it is worth confirming now, because if the answer is unwelcome the remedy is a different base and another training run.
The extraction model has the cleanest decision of the three, because reservation is not available to it at all. Weights trained outside Bedrock are not on the Provisioned Throughput eligibility list, so the real choice is managed import against self-hosting. If the architecture is one import supports, the per-minute meter suits research-shaped traffic well: model copies scale to zero after five idle minutes, and the standing cost is the monthly per-unit storage charge rather than a running endpoint. The cold start on the first call after a quiet period is the thing to check against the feature’s latency budget. If the architecture is not supported, or the team needs deployment control that managed serving does not expose, SageMaker takes it at instance-hours, which means designing for the quiet hours explicitly rather than discovering them on the invoice.
Across all three, the charges that are not inference deserve a line of their own in the forecast. Customising bills for the data processed and the passes over it, once. The resulting artefact bills monthly for as long as it exists. An imported model bills monthly for its stored weights, per Custom Model Unit it occupies. None of these follow traffic, so none of them shrink when a feature turns out to be unpopular, and all of them keep accruing after it is switched off unless somebody deletes the artefact.
Worked example
Take the classifier at a realistic volume and price both branches, because the arithmetic is what makes the constraint feel concrete. Say it handles thirty thousand documents a month, arriving in a weekday overnight batch that takes about ninety minutes, and each document runs roughly two thousand input tokens and two hundred output.
On the per-token branch, the bill is a multiplication: sixty million input tokens and six million output tokens a month, at the published per-token rates for that deployment. Nothing else accrues. Double the document count and the bill doubles; halve it and it halves. Finance can forecast this from a document count alone, which is the property that makes it easy to defend.
On the reserved branch, the arithmetic changes shape. The unit count comes from the busiest minute of that ninety-minute window, not from the monthly total, because the reservation has to be large enough for the peak it must absorb. Once sized, that unit bills for all seven hundred and thirty hours in the month, of which about thirty are doing work. The other seven hundred are the cost of the constraint. At that ratio, the monthly bill barely moves whether the classifier processes thirty thousand documents or three hundred thousand, which is worth understanding before treating it as a disaster: the reserved branch is expensive at this volume and would become competitive if throughput rose by an order of magnitude. The expensive part is the mismatch between a workload that runs ninety minutes a day and a meter that runs all day.
That comparison is why the base model chosen at the start of the training run is worth an hour of somebody’s attention. The two branches are not slightly different prices for the same thing; they are different relationships between usage and cost, and only one of them shrinks when the feature is quiet.
What’s worth remembering
- Provenance decides the option set: hosted weights, weights customised through Bedrock, and weights trained elsewhere each open a different set of serving surfaces, and the choice is made before there is a bill to look at.
- Provisioned Throughput eligibility is a published list of AWS-provided foundation models, covering those base models and Bedrock customisations of them; imported weights are not on it and cannot be reserved at all.
- On-demand serving of a customised model is limited to Nova Micro, Lite, 2 Lite and Pro and to Llama 3.3 70B Instruct, in two Regions, so confirm the base before the training run rather than after.
- The units differ in kind, not rate: per token, per reserved unit-hour, per active minute, per instance-hour. A price comparison is meaningless without a traffic shape.
- Anything metered on time bills through the quiet hours, which is why a workload that runs ninety minutes a day is the worst possible fit for a reservation and a good fit for anything that scales to zero.
- Training charges once and stored artefacts charge monthly until deleted, so a switched-off feature keeps costing something until somebody removes the model.