Spreading Bedrock Load with Cross-Region Inference Profiles

The situation

A B2B SaaS company serves three customer geographies from one AWS account. Bedrock runs the in-product AI features: a summarisation endpoint, an extraction endpoint, and a chat assistant, all on Claude Sonnet 5.

Claude Sonnet 5 is not offered as an in-Region model on the bedrock-runtime endpoint in commercial Regions. Its model card lists three geographic profile IDs and a global one, so every call already goes through an inferenceInferenceRunning a trained model to produce output – as opposed to training it. profile. The application was built in us-east-1, and it sends every tenant’s traffic through us.anthropic.claude-sonnet-5 from that Region.

Measured over the last quarter of production traffic:

• ThrottlingException on roughly 2% of peak-hour requests, and the rate grows month on month as adoption grows.
• Peak lands in the US/EU business-hours overlap.
• Customer distribution: roughly 50% US, 35% EU, 15% APAC, with the APAC share split between Sydney, Singapore and Tokyo accounts.
• EU contracts require EU processing. Today they do not get it.
• One product surface. The team wants one codebase, not three regional deployments.

Bedrock quotas on this endpoint are token-based, per model, per Region. Cross-Region InvokeModel tokens per minute for Claude Sonnet 5 is a separate quota from the on-demand single-Region one, and every tenant in the product is drawing on that one quota in that one source Region.

What actually matters

The throttling is a distribution problem. The model stays, the prompts stay, the application logic stays; what changes is the set of Regions allowed to serve a call, and whoever owns that decision also absorbs its operational weight. Per-Region health checks, retry ordering, version skew, an IAM policy that has to name every Region the traffic might reach. Bedrock will do the same job behind a single model ID, which leaves the team maintaining a lookup table instead of a dispatcher.

Residency is a property of the profile, and the boundary is not always the one the name suggests. For Claude Sonnet 5, the US geography keeps data in US and Canada Regions, the EU geography in EU Regions, the AU geography in Australia Regions. The global profile has no geographic boundary at all. An EU contract that says “processed in the EU” is satisfied by eu.; a US contract that a compliance reviewer reads as “processed in the United States” is not automatically satisfied by us., and that is worth checking before the renewal conversation rather than during it.

Geography coverage is per model, and the prefix you want may not exist. Claude Sonnet 5 publishes us., eu. and au. profiles and a global. one. There is no apac. profile for it, even though other models on Bedrock have one, so the Singapore and Tokyo tenants have no regional profile to route to. Read the model card rather than reasoning from another model’s prefixes.

Money runs the other way from the usual assumption. Geographic cross-Region inference adds no routing cost, and Bedrock prices the request from the Region you call, so spreading across a geography bills the same as calling in one Region. The global profile is priced roughly 10% below geographic on both input and output tokens. Quota relief is subtler than it sounds: the cross-Region TPM quota is per account, per model, per source Region, so routing spreads the compute across destination Regions without adding those Regions’ quotas together. More quota comes from a Service Quotas request, or from calling the profile from more than one source Region.

What we’ll filter on

Five filters.

1. Managed routing. One API call that Bedrock routes to a Region with capacity, with no per-request decision in the application.
2. A residency boundary that can be named and enforced at the routing layer, not in application code.
3. A profile published for this model in each geography the product sells into.
4. Token price at or below the current bill.
5. Low operational overhead. The two-engineer platform team cannot run a dispatcher service. Policy changes should be a config edit.

The landscape

Four shapes for spreading Claude Sonnet 5 load.

Geographic cross-Region inference profiles. A system-defined virtual model ID that takes a standard InvokeModel, Converse or streaming call and routes it to a destination Region in one geography. The ID is the base model ID with a geography prefix: us. across US and Canada Regions, eu. across EU Regions, au. across Australia Regions. Accounts in the GovCloud partition get a us-gov. profile spanning us-gov-west-1 and us-gov-east-1. A geographic profile’s destination list is fixed for the life of the profile. AWS reaches new Regions by publishing a new profile, not by widening an existing one.

The global cross-Region inference profile. global.anthropic.claude-sonnet-5 routes to supported commercial Regions worldwide and costs about 10% less per input and output token than the geographic profiles. Its destination set is every supported commercial Region, and it grows as AWS adds them. There is no global profile in GovCloud.

Manual spreading in the application. Hold a list of source Regions, pick one per request, catch ThrottlingException and try another. This still works with profile IDs, one client per source Region. The application then owns quota tracking, retry ordering, health checks and the failover logic.

Sticky per-tenant Region pinning. Send each tenant’s calls from a fixed source Region. Simple, and it does split the quota into per-Region buckets, but us-east-1 still carries the 50% that saturates first.

Capacity reservation is not selectable for this model, so it is not in the comparison below. Provisioned Throughput’s supported-model list stops several Claude generations short of Sonnet 5, inference profiles do not support Provisioned Throughput at all, and Sonnet 5’s card shows the Standard service tier only, with Priority, Flex and Reserved unsupported.

Evaluation

Side by side

Option	Managed routing	Named boundary	Covers all three geographies	Token price	Low ops
Geographic profiles (us., eu., au.)	✓	✓	✗	✓	✓
Global profile (global.)	✓	✗	✓	✓	✓
Manual spreading across source Regions	✗	✓	✓	✓	✗
Sticky per-tenant Region pinning	✗	✓	✗	✓	✓

No row ticks every column, and the gap is the same one in two places. Geographic profiles cover the US and EU tenants exactly, and leave Singapore and Tokyo with nothing regional to call. The global profile covers them and drops the residency boundary the EU contracts depend on. The two together cover the product, which is why the design below is split by tenant rather than picked once.

How the prefix routes the call

Three things worth spelling out.

The prefix selects a destination set, not a Region. A call to us.anthropic.claude-sonnet-5 looks the same at the SDK level as a call to any other model ID. Swap the prefix for eu. and the same client code routes across EU Regions instead, with no other change. That is what makes tenant-to-geography a lookup table rather than a branch at every call site.

Routing happens per request. Two consecutive calls to the same profile can run in different Regions, and the caller cannot pin one. For a stateless summarisation or extraction call that makes no difference. Prompt caching is the exception worth knowing: AWS notes that at times of high demand, cross-Region routing can lead to increased cache writes, so a cached prefix is not guaranteed to be reused on the next call. Sonnet 5 caches from 1,024 tokens per checkpoint, up to four checkpoints, with a 5-minute or 1-hour TTL, and cache reads bill at a lower rate than fresh input tokens. A chat assistant with a large system prompt should watch cacheWriteInputTokenCount after the switch.

The geographic boundary holds still. A geographic profile’s destination list is fixed, so us. will not acquire a new destination Region next quarter and there is no monthly audit to run. The global profile is the opposite: its destination set is every supported commercial Region and it widens as AWS adds them, which is the trade-off behind the lower token price.

The solution

Route by tenant residency, using two profile types rather than one.

US tenants call us.anthropic.claude-sonnet-5 from us-east-1. EU tenants call eu.anthropic.claude-sonnet-5, which also gives the EU contracts the processing boundary they were promised. Australian tenants call au.anthropic.claude-sonnet-5. The Singapore and Tokyo tenants have no regional profile for this model, so they call global.anthropic.claude-sonnet-5, which routes worldwide at roughly 10% below the geographic token rate. Where an APAC contract does bound processing geographically, the choice is the Australian profile or a different model with an apac. profile, and that is a contract conversation rather than a code change.

IAM is the part most likely to fail first. A geographic profile needs bedrock:InvokeModel on the inference-profile ARN in the calling Region, plus the foundation model ARN in the calling Region and in every destination Region, scoped with the bedrock:InferenceProfileArn condition. The global profile needs a third statement on the Region-agnostic arn:aws:bedrock:::foundation-model/anthropic.claude-sonnet-5, conditioned on aws:RequestedRegion of unspecified. Removing any one of those three denies global routing, which is also the documented way to turn it off deliberately.

Service Control Policies are the second thing to fail. A Region allowlist that omits a destination Region breaks the call even though the source Region is allowed, and a global profile needs unspecified allowed or a bedrock:InferenceProfileArn exception carved out of the Region deny.

Quota increases stay on the table and stack with the routing change. Cross-Region TPM is adjustable through Service Quotas in the source Region, and requesting an increase there is also the route to raising the on-demand and per-day quotas. AWS gives priority to accounts already consuming their allocation, so the request lands better after the traffic has been running than before.

Manual spreading across source Regions still has a use once those are in place, and it is the shape to reach for last. It puts quota tracking, retry ordering and failover in the application, which is a service to operate rather than a table to edit. Sticky per-tenant Region pinning is a residency tool, not a quota tool: use tenant geography to choose which profile, and let Bedrock choose the Region inside it.

Application inference profiles for cost allocation

System-defined profiles do the routing. Application inference profiles are created in the account with CreateInferenceProfile, take either a foundation model or a system-defined cross-Region profile as their modelSource, and carry tags. Those tags flow into cost allocation, so invocations show up in Cost Explorer and the Cost and Usage Report split by feature, tenant or environment.

Most production teams end up with one application inference profile per logical feature (summariser, extractor, chat assistant), each wrapping the system-defined profile that matches the model and geography. One caveat on the reporting side: Claude Sonnet 5 bills through AWS Marketplace, so the charges appear under the model provider rather than under Amazon Bedrock in Cost Explorer.

Worked example

• Model IDs. Tenant configuration maps each tenant to one of us., eu., au. or global. prefixed on anthropic.claude-sonnet-5. Nothing else in the call changes.
• IAM. The application role gets the profile ARNs, the foundation model ARN in every destination Region for the geographic profiles, and the Region-agnostic model ARN for the global one.
• SCP. The Region deny is amended to allow the destination Regions, or to exempt bedrock:InferenceProfileArn values matching the profiles in use.
• Cost allocation. Each feature wraps its profile in an application inference profile tagged Feature=<name> and Environment=<env>.
• Throttle handling. SDK exponential backoff on ThrottlingException stays as it is, and InvocationThrottles in the AWS/Bedrock namespace is the metric to watch it on.
• Observability. CloudWatch metrics for Bedrock carry a ModelId dimension and no Region-of-execution dimension, so the destination Region comes from CloudTrail in the source Region, in additionalEventData.inferenceRegion.
• Quota. Request an increase on Cross-Region InvokeModel tokens per minute for Claude Sonnet 5 in us-east-1 and eu-west-1, after a fortnight of the new traffic pattern.

The throttling ends up fixed by a change to the model-ID string in tenant configuration, an IAM policy update and an SCP amendment. No dispatcher service, no Route 53 record, no per-Region quota tracker, and the EU residency gap closes in the same change.

What’s worth remembering

1. Cross-Region inference profiles are the Bedrock-native way to spread load. Prefix the model ID with a geography, us., eu., au. or us-gov. for Claude Sonnet 5, and Bedrock selects a destination Region with capacity per request.
2. Which prefixes exist is a per-model fact on the model card. Claude Sonnet 5 has no apac. profile, so APAC tenants outside Australia route through global. or not at all.
3. Geographic profiles bill at the normal on-demand rate with no routing charge, priced from the Region you call. The global profile bills about 10% less per input and output token and gives up the geographic boundary to do it.
4. A geographic profile’s destination list never changes; the global profile’s set of Regions widens as AWS adds commercial Regions. Only one of those needs watching for residency.
5. Cross-Region tokens per minute is its own quota, per model and per source Region, so routing spreads compute rather than summing the destinations’ quotas. More headroom comes from Service Quotas, or from calling from a second source Region.