The situation
A company runs a customer-facing assistant on Amazon Bedrock in a single region. The stack is familiar: an application tier behind an API, a Claude model invoked on demand, a guardrail that filters both the prompt and the response, and a Knowledge Base for retrieval, backed by a vector store and fed from a bucket of source documents. It works well on a normal day.
Two things have started to hurt. First, at peak the on-demand model calls hit the account’s per-region throughput quota and start returning throttling errors, so users see slow or failed responses exactly when traffic is highest. Second, the whole assistant lives in one region, and a recent multi-hour Bedrock disruption in that region took the feature completely offline with no fallback. Leadership now wants an availability target the single-region design can’t meet, and the throttling has to stop being a peak-hour lottery.
There’s a constraint sitting underneath both problems. Some of the traffic carries customer data that, by contract, has to stay within a defined geography. So any answer that spreads load or fails over to other regions has to respect where those requests are allowed to be processed, not just where they happen to start.
What actually matters
Resilience for a model-backed service is not one problem. There’s the throttling problem, which is about capacity and happens on ordinary days, and there’s the regional-failure problem, which is about survival and happens rarely but totally. They have different fixes, and it’s worth not conflating them: raising effective throughput does nothing for a region outage, and a warm standby region does nothing for a quota you hit at 2pm every Tuesday.
The throttling side is mostly about spreading invocations. On-demand model access has per-region account quotas, and a single region gives you exactly one bucket of capacity. Bedrock’s cross-region inference profiles let a single invocation be routed to one of several regions within a geography, so the effective ceiling is the sum of that geography’s capacity rather than one region’s, and transient spikes in one region get absorbed by the others. That directly smooths throttling. It also carries a data-residency consequence worth stating plainly: a request submitted in one region may be processed in another region within the same profile’s geography. If the contract says data stays in a defined area, the profile’s geography has to sit inside that area, and requests that can’t leave a specific region can’t use a profile that would route them out of it.
The regional-failure side is about having somewhere to go when a region is gone. That means the second region has to be genuinely ready, not a diagram. Model access has to be enabled there for the exact models you call. The guardrail has to exist there too, because guardrails are regional and don’t follow you across the boundary; the same is true of the prompts and the Knowledge Base. A failover region that can take app traffic but can’t invoke your model, or invokes it without the guardrail, isn’t a failover region, it’s an incident with extra steps.
Retrieval is the part people forget. The answer quality depends on the Knowledge Base, and a Knowledge Base is regional: the vector store lives in one region and so does the ingestion pipeline. Failing over the app and the model to a second region that has an empty or stale index gives you a service that runs and answers badly. Making retrieval survive a region loss means the source documents have to be present in the second region and the vector index has to be built and kept current there, which is a replication job for the source bucket plus a standing Knowledge Base and vector store on the other side.
And model availability differs by region. Not every model, and not every feature of a model, is offered everywhere, and new models often land in a subset of regions first. The failover region is only viable if it actually supports the models and features you depend on. That constraint can decide which region you pair with, and sometimes it’s the thing that forces a model choice rather than the other way round.
The last thing that actually matters is the number that governs the architecture: how much downtime and how much data loss you can tolerate. A tight recovery-time objective, seconds to a minute or two, pushes you toward active-active, both regions live and serving, because there’s no time to spin anything up. A looser one tolerates active-passive, a warm standby you promote when the primary fails. Active-active roughly doubles the steady-state cost and the operational surface; active-passive is cheaper but has a real, non-zero recovery time and needs the standby kept warm enough to trust. The recovery objectives, the residency rule, and the cost of a second region are the three levers, and they trade against each other.
What we’ll filter on
- Availability target: what recovery-time and recovery-point objectives does the service actually have to hit?
- Failure mode covered: peak-hour throttling, full regional outage, or both?
- Data residency: which geography or region are the requests allowed to be processed in?
- Component readiness: are model access, guardrails, prompts, and the Knowledge Base present and current in the second region?
- Retrieval durability: is the source data replicated and the vector index kept current across regions?
- Cost and operational load: what does keeping the second region warm, or live, cost in money and in things to run?
The resilience landscape
Single region, on-demand. The starting point. One region’s quota, one region’s fate. Simplest to run, cheapest, and the residency story is trivial because nothing leaves. It can’t meet a demanding availability target and it caps out at one region’s throughput.
Provisioned Throughput. Buy dedicated model capacity in a region to remove on-demand throttling for a committed, predictable load. It fixes the capacity problem for steady high volume and gives consistent latency, but it’s a regional commitment with a cost floor, it doesn’t help with a region outage, and it doesn’t stretch across regions the way a cross-region profile does. It suits a known, sustained baseline rather than spiky traffic.
Cross-region inference profiles. Invoke through a profile instead of a single model ID, and Bedrock routes each call to one of several regions in a geography, raising the effective throughput ceiling and absorbing spikes so throttling smooths out. This is the direct answer to the peak-hour capacity problem, and it needs no standby infrastructure of your own. The catch is residency: a request can be processed in another region within the geography, so the profile has to stay inside the area your data is allowed to be in.
Active-passive (warm standby) across regions. A second region kept ready with model access, the guardrail, the prompts, and a current Knowledge Base, but not taking live traffic until the primary fails, at which point routing shifts and the standby is promoted. This survives a full regional outage at a moderate cost, with a recovery time measured in the minutes it takes to detect and cut over. It needs the standby kept genuinely warm, and it needs the replication running continuously so the index isn’t stale when you land on it.
Active-active across regions. Both regions serve live traffic all the time, fronted by health-checked routing so a failure just stops sending traffic to the sick region. This meets the tightest recovery objectives because there’s nothing to promote, and it doubles as capacity headroom. The price is roughly double the steady-state cost and two live environments to keep in step: guardrails, prompts, and Knowledge Bases have to stay identical on both sides, which is real operational discipline.
Front-door routing (Route 53 / Global Accelerator). Not a standalone answer but the mechanism that makes failover work: health checks that detect a failed region and routing policies (failover for active-passive, latency or weighted for active-active) that steer users to a healthy endpoint. Whichever cross-region posture you pick, this is how traffic actually moves.
Side by side
| Option | Fixes throttling | Survives region outage | Recovery time | Residency control | Steady-state cost |
|---|---|---|---|---|---|
| Single region, on-demand | ✗ | ✗ | N/A (down) | Full (nothing leaves) | Lowest |
| Provisioned Throughput | ✓ (steady load) | ✗ | N/A (down) | Full (one region) | High (committed) |
| Cross-region inference profiles | ✓ | ✗ (invocation only) | N/A | Geography-bounded | Low (usage-based) |
| Active-passive (warm standby) | Partial | ✓ | Minutes (promote) | You choose regions | Medium |
| Active-active | ✓ | ✓ | Seconds (no promote) | You choose regions | Highest (~2x) |
| Front-door routing | ✗ | Enables it | Drives the cutover | Neutral | Low |
The table splits the two problems cleanly. A cross-region inference profile answers throttling but not outage, because it only spreads the model invocation, not your app tier or your retrieval. Active-passive and active-active answer outage; which one depends entirely on the recovery-time objective and what you’ll pay. And they compose: a real design usually runs a cross-region profile for throughput inside whichever multi-region posture the availability target demands.
The picks in depth
Start by fixing throttling, because it’s the cheaper problem and it doesn’t require a second environment of your own. Switch the model invocation from a single model ID to a cross-region inference profile whose geography covers the regions you’re entitled to use. Each call is then routed across that geography’s capacity, the effective throughput ceiling rises, and the peak-hour throttling errors smooth out without you provisioning anything. Two caveats decide whether this is safe. The residency one is first: a request may be processed in a different region within the geography, so if a class of traffic is contractually pinned to one region, it can’t ride a profile that would route it elsewhere, and you keep that traffic on a direct in-region invocation. The second is that a cross-region profile addresses the model call only. It does nothing for a region outage, because your app tier, guardrail, and Knowledge Base still live in one place.
For surviving a region outage, the recovery-time objective picks the posture. If the target is minutes and the budget is moderate, build active-passive: a second region kept warm with model access enabled for the exact models you invoke, the guardrail recreated there (guardrails are regional and don’t replicate, so you create and version the same configuration on both sides), the prompts present, and a standing Knowledge Base with its own vector store. Route 53 health checks watch the primary and a failover routing policy shifts users across when it goes unhealthy. The discipline that makes this real is keeping the standby current: continuously, not on the morning of the incident.
If the target is tighter than a promote-and-cut-over can hit, go active-active: both regions serve live traffic behind latency or weighted routing, and a failed region simply stops receiving requests. There’s nothing to promote, so recovery is effectively the time for health checks to react. You pay for it in roughly doubled steady-state cost and in keeping two live environments identical: the same guardrail configuration, the same prompt versions, and Knowledge Bases that answer the same way on both sides. Drift between the two is the failure mode here, so the guardrail, prompt, and ingestion config want to be deployed from the same source of truth rather than clicked into place twice.
Whichever outage posture you choose, retrieval has to come along or the failover answers badly. Replicate the Knowledge Base’s source bucket to the second region with S3 Cross-Region Replication so the documents are present, and stand up a Knowledge Base and vector store there that ingests from the replicated source, so the index is built and current on the other side. Newly added documents replicate to the second region and get ingested into its index, keeping the recovery-point gap down to the replication and ingestion lag rather than a full rebuild. And confirm, before committing to a region pairing, that the second region actually offers the models and the features you depend on; model and feature availability varies by region, and a standby that can’t run your model is not a standby. That availability check sometimes drives the whole decision, including which model you standardise on.
A worked example: from one region to a resilient pair
The assistant starts in a single region: app tier, on-demand Claude invocation, one guardrail, one Knowledge Base over an OpenSearch Serverless vector store fed from an S3 bucket. Its contract says customer data must stay within one geography, its new availability target allows a couple of minutes of recovery, and peak traffic throttles the model calls.
The throttling fix lands first. The app stops calling a single model ID and calls a cross-region inference profile scoped to that geography, so invocations spread across the geography’s regions and the peak-hour throttling clears. Because the profile’s geography sits inside the contractual area, residency holds; the requests may be processed in a neighbouring region, but never outside the boundary the contract sets.
The outage fix is active-passive, because minutes of recovery is acceptable and it’s the cheaper posture. A second region in the same geography gets model access enabled for the same model, the guardrail recreated with the identical configuration and version, the prompt library deployed, and a Knowledge Base with its own vector store. S3 Cross-Region Replication copies the source documents across, and the second region’s Knowledge Base ingests them so its index stays current. Route 53 health-checks the primary endpoint and a failover routing policy points at the standby. The recovery-point gap is whatever the replication and ingestion lag is; the recovery-time is detection plus the failover routing switch. When the primary region has its next bad hour, traffic moves to a standby that has the model, the guardrail, the prompts, and a warm index, and the assistant keeps answering, correctly, from inside the geography it’s allowed to run in.
What’s worth remembering
- Throttling and regional outage are two problems: spreading invocations fixes capacity, a second region fixes survival, and neither fix touches the other.
- Cross-region inference profiles raise effective throughput and smooth throttling by routing each call across a geography’s regions, with no standby of your own to run.
- A profile can process a request in another region within its geography, so pin the profile’s geography inside the area your data is contractually allowed to be in.
- Guardrails, prompts, and Knowledge Bases are regional and don’t follow you; a failover region has to have them recreated and kept current, not just the app tier.
- Retrieval has to survive too: replicate the source bucket across regions and keep a standing Knowledge Base and vector store so the failover index isn’t empty or stale.
- Model and feature availability differs by region, so confirm the second region actually supports the models you invoke before committing to the pairing, and let that constrain the model choice if it must.
- The recovery-time objective chooses the posture: active-passive for minutes of promote-and-cut-over, active-active for near-zero because there’s nothing to promote.
- Active-active roughly doubles steady-state cost and demands both regions stay identical; drift in guardrails, prompts, or index config is its main failure mode.
- Route 53 health checks with failover, latency, or weighted routing are the mechanism that moves users to a healthy region; the posture decides the policy.
- Recovery objectives, data residency, and the cost of a warm or live second region are the three levers, and a real design usually runs a cross-region profile for throughput inside whichever multi-region posture the availability target demands.