Cheat Sheet: Integration and Deployment

Fast revision for deployment surfaces, delivery modes, API design and integration patterns, resilience, and routing. The companion sheet on model-decided control flow is agents and orchestration; everything here is the plumbing around it. Skim the tables, drill the decision rules, watch the traps.

Deployment surfaces

Where the model runs, and what each surface bills you for. Argued in full in choosing an inference option.

Surface	Latency	Cost shape	Pick it when
Lambda invoking Bedrock on demand	Milliseconds of overhead on the call; 15-minute ceiling per invocation	Per request plus duration; nothing while idle	Serverless computing suits the caller: event-driven or spiky traffic, and no compute bill between bursts
Bedrock on-demand	Interactive, subject to account throttling under contention	Per input and output token, no commitment	The default for variable interactive volume
Bedrock Provisioned ThroughputProvisioned ThroughputReserved Bedrock capacity bought by the hour for a fixed term, paid for whether traffic fills it or not.	Interactive with a guaranteed floor	Per hour, per model unitModel unitThe billing block Provisioned Throughput is sold in – one unit delivers a fixed tokens-per-minute rate for a specific model., committed	Volume is steady and high, or a fine-tuned model has no on-demand path
Bedrock batch inferenceBatch inferenceSubmitting a bulk job of model calls to run asynchronously at a lower per-token price, trading immediacy for cost.	Hours; the job runs on its own schedule	Roughly half on-demand per token	Bulk offline work with nobody waiting
SageMaker AI endpoint	Lowest and most consistent for a model you host	Per instance-hour, idle included	The model is not a Bedrock FM: open weights, an imported model, or a hybrid split across both
Self-hosted container on ECS or EKS with Fargate	Whatever the image and instance give you	Per task or node hour, GPU included	You need the serving stack, the memory profile, or the accelerator choice under your own control

The last row is container orchestration territory. Deploying a large language model in a container is not the traditional ML deployment with a bigger instance. Weights are measured in tens of gigabytes, so model loading strategies decide cold-start time; GPU utilisation and token processing capacity decide throughput far more than vCPU count; and memory requirements have to cover the weights plus the KV cache for every concurrent sequence. Size the task for concurrency and context length, not for request rate.

Delivery modes

How the answer reaches the caller. Argued in full in sync, async, or streaming and, for the bulk end, processing documents asynchronously.

Mode	Transport	Timeout risk	API Gateway integration
Synchronous Converse or InvokeModel	One request, one response, connection held for the whole generation	High: the integration timeout sits around 29 to 30 seconds and Lambda caps at 15 minutes	REST or HTTP API with a Lambda proxy; buffering is harmless here
Streaming with ConverseStream or InvokeModelWithResponseStream	Chunks as generated, over server-sent events or a socket	Time to first token drops; total generation time is unchanged	WebSocket API pushing chunks, chunked transfer encoding, or Lambda response streaming through a function URL; a buffering REST integration cancels the benefit
Asynchronous job	Acknowledge immediately, run the work on SQS or Step Functions, write the result to S3 or a database	None on the request path	Return 202 with a job id; the client polls a status route or gets a push over WebSockets or AppSync
Batch inference	S3 manifest in, S3 out	None, replaced by job scheduling latency	Not fronted by API Gateway at all; submit the job and collect

Integration patterns

Four shapes for enterprise system integration, and a real deployment usually runs more than one side by side. Argued in full in wiring an assistant into systems you cannot change.

Pattern	Freshness	Coupling	Load on the source
Synchronous API call through API Gateway and Lambda	Current to the millisecond	Tight: no answers during the source’s maintenance window, and every latency spike is inherited	One call per user question; needs a usage-plan ceiling to survive a runaway loop
EventBridge event with an SQS buffer	Seconds behind the change	Loose: the source emits and stops caring who listens	One publish per change; a dead-letter queue catches poison messages, archive and replay fills a missed window
Scheduled sync into S3, then a knowledge base over the bucket	As of the last sync	Loose and one-way	One scheduled read, which is what a fragile system can actually survive
Inbound webhook on API Gateway with a Lambda handler	Seconds, when the source chooses to push	Loose, but the source owns the initiative	None: the push is on the source’s schedule, not yours

The scheduled-sync row splits by where the data lives. Amazon AppFlow moves records from SaaS sources such as a CRM, with field mapping configured rather than coded. AWS DataSync moves files from on-premises NFS and SMB shares. AWS Transfer Family puts a managed SFTP, FTPS or AS2 endpoint in front of S3 for a partner or a legacy job that can only drop a file somewhere.

Event-driven architectures are the shape to reach for whenever the integration can tolerate seconds of lag, because they are the only one of the four where adding a second consumer costs the source nothing.

Two variations are worth naming. Hybrid cloud architectures come up when data cannot leave a building rather than a country: AWS Outposts puts an AWS-managed rack inside the boundary and AWS Wavelength puts compute at the carrier edge, and neither runs Bedrock, so the local side holds the data and the sanitising step while the Region holds the model. Serving a feature when the data cannot leave walks the whole split. And a centralised facade in front of Bedrock, whether an API Gateway REST API over a Lambda proxy or a container behind an Application Load Balancer, is how an organisation gets per-team throttling and mandatory guardrails that individual teams cannot opt out of; a GenAI gateway covers what each shape costs you.

CI/CD for AI applications is the fourth piece of the enterprise story. CodePipeline stages with CodeBuild actions run the deterministic tests, the security scans and the evaluation jobs; CodeDeploy shifts an alias in a canary or linear pattern with rollback on a CloudWatch alarm; and CDK or CloudFormation carries prompts, guardrail configuration and action-group schemas as versioned source. Building the pipeline sets out the stage sequence and the two gates.

Resilience levers

What each one actually fixes. Argued in full in handling throttling and rate limits and, for the classification of errors, which Bedrock errors to retry.

Lever	Failure it addresses	Notes
SDK exponential backoff with jitter	Transient ThrottlingException and retryable server errors	Standard retry mode backs off; adaptive adds client-side rate limiting. Adds no capacity, so it cannot fix a structurally over-quota workload
API Gateway usage plans and stage throttling	One caller flooding a shared ceiling, or an agent loop that will not stop	Rate, burst and quota per API key, enforced at the edge before any compute runs
Circuit breaker	A dependency that is down staying down while retries pile onto it	Open after a failure threshold, fail fast, probe with a half-open call. Keeps one sick tool from stalling every request
Fallback model	Sustained scarcity, or one model unavailable in one region	The fallback needs to be good enough for the degraded path, and you need a written rule for what degrades
Cross-region inference profileCross-region inferenceLetting a request be served from any of several regions, raising effective throughput and riding out pressure in one of them.	Load concentrating on a single region’s quota	Raises effective throughput rather than guaranteeing a floor; check where the data is permitted to be processed

X-Ray sits across all of them: it is what tells you whether the latency came from the model, the tool, or the retry loop in between.

Routing shapes

What a request costs once you stop sending everything to the same model. Argued in full in routing between a cheap and a capable model.

Shape	Who decides	Cost multiplier
Static configuration	You, ahead of time, in application code or AppConfig	One model call, and no way to react to a hard request
Step Functions content-based routing	A deterministic rule in the state machine, on task type, length or a small classifier’s label	One model call, plus the classifier’s if one runs
Bedrock Intelligent Prompt Routing	Bedrock, per request, within one model family	Below one call’s worth on a suitable mix, with no separate charge for routing; you pay for whichever family member serves
Cascade	The cheap model’s own confidence, or a validator on its answer	One cheap call on the easy majority; cheap plus capable on every escalation, so the hard tail pays twice
Ensemble	Aggregation logic over several answers	Every model in the set, plus the aggregation step

Who does what

Thing	What it is	Reach for it when
Strands Agents	An open-source SDK for building an agent in code, with a model-driven loop	You are writing the agent yourself and want the loop, tools and memory handled by a library
AWS Agent Squad	An open-source framework for multi-agent systems: a classifier routes to a specialist and shared context follows the conversation	Work splits into distinct specialisms and one of them has to answer each turn
Amazon Bedrock AgentCore	A framework-agnostic production runtime with memory, a gateway, identity and observability	An agent you already wrote needs to run in production
Model Context Protocol (MCP)	The open protocol between an agent and its tools	You want one tool surface that several agents and several frameworks can all read
Amazon Q Developer	An assistant for code and your AWS account, in the IDE, CLI and console	An engineer needs help writing, reviewing or explaining code
Kiro	An agentic development environment across an IDE and a CLI, built for spec-driven development	Work should start from a specification the agent plans against rather than a blank completion
Amazon Quick	A finished, managed assistant over enterprise data, honouring each asker’s permissions, with Flows for automation, Quick Sight for BI and Spaces for shared knowledge	Staff need answers from internal systems and nobody wants to build a product
Amazon Q Business	The earlier name for that enterprise assistant, still listed in the exam guide	You see the old name and need to recognise it as today’s Amazon Quick

The three finished products are set against the platform in Kiro, Amazon Quick, or Bedrock.

Decision rules

• If traffic is spiky and the compute layer should cost nothing between bursts, then invoke Bedrock from Lambda on demand.
• If volume is steady and high, or a fine-tuned model has no on-demand path, then reserve Provisioned Throughput in model units.
• If nobody is waiting and the pile is large, then submit a batch inference job against an S3 manifest.
• If the model is not a Bedrock FM, then it belongs on a SageMaker AI endpoint or a container you run, and the hybrid case puts some of each in the same architecture.
• If a container serves the model, then size it for weights plus KV cache at your concurrency, and treat model load time as part of the scale-out latency.
• If a human is watching the answer appear, then stream it and pick a transport that forwards chunks instead of buffering them.
• If the completion can run past thirty seconds, then get it off the request path: acknowledge, queue, and deliver the result out of band.
• If a REST integration buffers and you still need streaming, then move to a WebSocket API or to Lambda response streaming through a function URL, and accept that usage plans do not follow you.
• If an answer must be current to the second, then call the source synchronously and put a usage plan in front of it.
• If seconds of lag are acceptable and more than one consumer may appear, then publish to EventBridge and buffer with SQS.
• If the source is fragile and cannot take per-question traffic, then sync a copy into S3 on a schedule and index that.
• If the source can push but cannot be polled, then take a webhook: verify the signature, write to SQS, and return 200 before doing slow work.
• If the data cannot leave a specific site, then put the storage and the redaction step on Outposts or at a Wavelength edge and send only sanitised text to the Region.
• If several teams share one account’s quota, then a gateway with usage plans is the only place a per-team ceiling can be enforced.
• If a call fails with throttling, then back off with jitter before anything else; if it keeps failing, the workload is over quota and needs capacity, not retries.
• If a dependency is failing every call, then open a circuit breaker so requests fail fast instead of queueing behind a dead service.
• If prompts vary in difficulty within one model family, then point the request at an Intelligent Prompt Routing router and set the response-quality tolerance.
• If the easy majority is large and a reliable weakness signal exists, then cascade; if it does not, route on a rule you can read.
• If a release changes a prompt, a guardrail or a model id, then it goes through the same pipeline as code, with an evaluation gate before the deployment gate.

Traps

• A synchronous call behind API Gateway is capped by the integration timeout, not by the model. A completion that grew from twenty seconds to thirty-five starts returning 504 without anything in the model changing.
• Streaming does not make generation faster. It moves the first token forward; total time is the same, so a slow model is still slow.
• A standard REST or HTTP API integration buffers the whole response. Calling ConverseStream behind one gets you streaming to the Lambda and a single blob to the browser.
• Lambda’s fifteen-minute ceiling is per invocation, not per workflow. Long chains split across steps; they do not get a longer function.
• Provisioned Throughput bills by the hour whether traffic arrives or not, so reserved capacity that sits idle overnight is still charged.
• Batch inference is cheap because nobody is waiting. It is never the answer when a user is watching a spinner.
• SQS visibility timeout has to exceed worst-case model latency, or a second worker picks up a document the first is still processing and you pay twice for a duplicate answer.
• A webhook handler that is not idempotent on the provider’s delivery id will apply the same event twice, because deliveries retry and arrive out of order.
• Retries without backoff make throttling worse. Synchronised retries from many callers arrive as one spike against the same ceiling.
• A cross-region inference profile raises throughput and changes where inference happens. Check the residency rules before enabling it.
• A cascade that escalates most requests costs more than sending everything to the capable model. Measure the escalation rate before shipping it.
• Intelligent Prompt Routing routes within a model family, not across arbitrary models, so it fits when one family spans the capability range you need.
• An ensemble multiplies token spend by the number of models. It buys reliability on high-stakes answers and wastes money on everything else.
• Reserved concurrency on the worker is what keeps Lambda from scaling straight past the Bedrock quota. Without it, the fan-out that fixed your backlog creates the throttling.
• Amazon Q Developer and Amazon Quick sound alike and do different jobs. One helps with code and your AWS account, the other answers from company data with citations.

Say it in one line

1. Lambda on demand for spiky callers, Bedrock on-demand as the default, Provisioned Throughput for steady high volume, batch for bulk with nobody waiting, SageMaker AI endpoints and containers for models Bedrock does not serve.
2. An LLM in a container is sized by weights plus KV cache, GPU utilisation and token throughput, and its cold start is dominated by model loading.
3. Synchronous holds a connection and inherits every timeout; streaming fixes the first-token feel through a transport that forwards chunks; asynchronous takes the work off the request path; batch trades immediacy for about half the token price.
4. Streaming needs a WebSocket API, chunked transfer encoding, or Lambda response streaming, because a standard REST integration buffers.
5. Four integration shapes: synchronous call for freshness, EventBridge for loose coupling, scheduled sync through AppFlow, DataSync or Transfer Family for a fragile source, and a webhook when the source can only push.
6. AppFlow is SaaS records, DataSync is on-premises file shares, Transfer Family is an SFTP, FTPS or AS2 drop into S3.
7. Outposts and Wavelength hold data and pre-processing on site; Bedrock runs in the Region, so only sanitised text crosses the line.
8. A gateway in front of Bedrock is where per-team rate limits and mandatory guardrails become compulsory rather than optional.
9. Backoff with jitter for transient throttles, usage plans for noisy callers, circuit breakers for dead dependencies, fallback models for scarcity, cross-region profiles for regional pressure, X-Ray to see which one fired.
10. Retries add no capacity; a workload structurally over quota needs a quota increase, Provisioned Throughput, or queued and deferred work.
11. Routing runs from static configuration through content-based rules and Intelligent Prompt Routing to cascades and ensembles, and the cost multiplier climbs with the number of models each request touches.
12. Prompts, guardrails and model ids ship through CodePipeline, CodeBuild, CodeDeploy and CDK like any other source, with an evaluation gate ahead of the deployment gate.
13. Strands builds the agent, Agent Squad coordinates several, AgentCore runs them in production, MCP is the protocol between agent and tools.
14. Q Developer is for code and your AWS account, Kiro is spec-driven development, Amazon Quick (formerly Amazon Q Business) is the finished assistant over enterprise data.