Cheat Sheet: Integration and Deployment

Fast revision for deployment surfaces, delivery modes, API design and integration patterns, resilience, and routing. The companion sheet on model-driven control flow is agents and orchestration; everything here is the plumbing around it.

Deployment surfaces

Where the model runs, and what each surface bills you for. Argued in full in choosing an inference option.

Surface	Latency	Cost shape	Pick it when
Lambda invoking Bedrock on demand	Milliseconds of overhead on the call. 900-second ceiling per invocation	Per request plus duration. Nothing while idle	Traffic is event-driven or spiky, and no compute should be billed between bursts
Bedrock on-demand	Interactive, subject to account throttling under contention	Per input and output token, no commitment	The default for variable interactive volume
Bedrock Provisioned ThroughputProvisioned ThroughputReserved Bedrock capacity bought by the hour for a fixed term, paid for whether traffic fills it or not.	Interactive with a guaranteed floor	Per hour, per model unitModel unitThe billing block Provisioned Throughput is sold in – one unit delivers a fixed tokens-per-minute rate for a specific model., on a no-commitment, one-month or six-month term	Volume is steady and high, or the model has been customised in Bedrock, which leaves no on-demand path
Bedrock batch inferenceBatch inferenceSubmitting a bulk job of model calls to run asynchronously at a lower per-token price, trading immediacy for cost.	Hours. The job runs on its own schedule	Half the on-demand token price	Bulk offline work with nobody waiting
SageMaker AI endpoint	Lowest and most consistent for a model you host	Per instance-hour, idle included	The model is not a Bedrock FM: open weights, an imported model, or a hybrid split across both
Self-hosted container on ECS or EKS	Whatever the image and instance give you	Per task or node hour, accelerator included	You need the serving stack, the memory profile, or the accelerator choice under your own control

The last row is container orchestration territory. A container serving a large language model is sized differently from a traditional ML container on a bigger instance. Weights run to tens of gigabytes, so the loading strategy sets cold-start time. GPU utilisation and token throughput set capacity far more than vCPU count, and memory has to cover the weights plus the KV cache for every concurrent sequence. Fargate has no GPU support, so accelerated serving needs EC2 capacity in the cluster. Size the task for concurrency and context length, not for request rate.

Delivery modes

How the answer reaches the caller. Argued in full in sync, async, or streaming and, for the bulk end, processing documents asynchronously.

Mode	Transport	Timeout risk	API Gateway integration
Synchronous Converse or InvokeModel	One request, one response, connection held for the whole generation	High. A REST API’s integration timeout runs from 50 milliseconds to 29 seconds by default, and Lambda caps at 15 minutes	REST or HTTP API with a Lambda proxy. Buffering is harmless here
Streaming with ConverseStream or InvokeModelWithResponseStream	Chunks as generated, over server-sent events or a socket	Time to first token drops. Total generation time is unchanged	A REST API with the proxy integration’s response transfer mode set to STREAM, a WebSocket API, or Lambda response streaming through a function URL. The default BUFFERED mode cancels the benefit
Asynchronous job	Acknowledge immediately, run the work on SQS or Step Functions, write the result to S3 or a database	None on the request path	Return 202 with a job id. The client polls a status route or receives a push over WebSockets or AppSync
Batch inference	S3 manifest in, S3 out	None, replaced by job scheduling latency	Not fronted by API Gateway at all. Submit the job and collect

Response streaming on a REST API applies to AWS_PROXY and HTTP_PROXY integrations only, runs for up to 15 minutes, and rules out endpoint caching, content encoding and VTL response mapping. It also sidesteps the 29-second timeout and the 10 MB response payload limit. HTTP APIs have no equivalent setting. The 29-second maximum itself can be raised on Regional and private REST APIs through a quota request, which may require reducing the Region-level throttle quota on the account; edge-optimized APIs are stuck at 29.

Integration patterns

Four shapes for enterprise system integration, and a real deployment usually runs more than one side by side. Argued in full in wiring an assistant into systems you cannot change.

Pattern	Freshness	Coupling	Load on the source
Synchronous API call through API Gateway and Lambda	Current to the millisecond	Tight. No answers during the source’s maintenance window, and every latency spike is inherited	One call per user question. Needs a usage-plan ceiling to survive a runaway loop
EventBridge event with an SQS buffer	Seconds behind the change	Loose. The source emits once and consumers subscribe independently	One publish per change. A dead-letter queue catches poison messages, archive and replay fills a missed window
Scheduled sync into S3, then a knowledge base over the bucket	As of the last sync	Loose and one-way	One scheduled read, which is what a fragile system can sustain
Inbound webhook on API Gateway with a Lambda handler	Seconds, when the source pushes	Loose, but the source controls when data arrives	None. The push runs on the source’s schedule, not yours

The scheduled-sync row splits by where the data lives. Amazon AppFlow moves records from SaaS sources such as a CRM, with field mapping configured rather than coded. AWS DataSync moves files from on-premises NFS, SMB, HDFS and object storage. AWS Transfer Family puts a managed SFTP, FTPS, FTP or AS2 endpoint in front of S3 or EFS for a partner or a legacy job that can only drop a file somewhere.

Event-driven architectures are the shape to reach for whenever the integration can tolerate seconds of lag, because they are the only one of the four where adding a second consumer leaves the source untouched.

Two variations are worth naming. Hybrid cloud architectures come up when data cannot leave a building rather than a country. AWS Outposts puts an AWS-managed rack inside the boundary and AWS Wavelength puts compute at the carrier edge; neither runs Bedrock, so the local side holds the data and the sanitising step while the Region holds the model. Serving a feature when the data cannot leave walks the whole split. A centralised facade in front of Bedrock, whether an API Gateway REST API over a Lambda proxy or a container behind an Application Load Balancer, is how an organisation gets per-team throttling and mandatory guardrails that individual teams cannot opt out of. A GenAI gateway covers what each shape costs you.

CI/CD for AI applications is the fourth piece of the enterprise story. CodePipeline stages with CodeBuild actions run the deterministic tests, the security scans and the evaluation jobs. CodeDeploy shifts an alias in a canary or linear pattern with rollback on a CloudWatch alarm, and CDK or CloudFormation carries prompts, guardrail configuration and action-group schemas as versioned source. Building the pipeline sets out the stage sequence and the two gates.

Resilience levers

What each one actually fixes. Argued in full in handling throttling and rate limits and, for the classification of errors, which Bedrock errors to retry.

Lever	Failure it addresses	Notes
SDK exponential backoff with jitter	Transient ThrottlingException and retryable server errors	Standard retry mode backs off; adaptive adds client-side rate limiting. Adds no capacity, so it cannot fix a structurally over-quota workload
API Gateway usage plans and stage throttling	One caller flooding a shared ceiling, or an agent loop that keeps calling	Rate, burst and quota per API key, enforced at the edge before any compute runs. REST APIs only
Circuit breaker	A dependency that is down staying down while retries pile onto it	Open after a failure threshold, fail fast, probe with a half-open call. Keeps one failing tool from stalling every request
Fallback model	Sustained scarcity, or one model unavailable in one Region	The fallback needs to be good enough for the degraded path, and you need a written rule for what degrades
Cross-Region inference profileCross-region inferenceLetting a request be served from any of several regions, raising effective throughput and riding out pressure in one of them.	Load concentrating on a single Region’s quota	Raises effective throughput rather than guaranteeing a floor. No extra routing charge, but check where the data is permitted to be processed, and note that inference profiles do not support Provisioned Throughput

X-Ray sits across all of them: it is what tells you whether the latency came from the model, the tool, or the retry loop in between.

Routing shapes

What a request costs once you stop sending everything to the same model. Argued in full in routing between a cheap and a capable model.

Shape	What selects the model	Cost multiplier
Static configuration	You, ahead of time, in application code or AppConfig	One model call, and no way to react to a hard request
Step Functions content-based routing	A deterministic rule in the state machine, on task type, length or a small classifier’s label	One model call, plus the classifier’s if one runs
Bedrock Intelligent Prompt Routing	A Bedrock router, per request, across two models in one family	Whichever family member serves, plus USD$1 per 1,000 routed requests
Cascade	A confidence score from the cheap model, or a validator on its answer	One cheap call on the easy majority. Cheap plus capable on every escalation, so the hard tail costs two calls
Ensemble	Aggregation logic over several answers	Every model in the set, plus the aggregation step

Who does what

Thing	What it is	Reach for it when
Strands Agents	An open-source SDK for building an agent in code, with a model-driven loop	You are writing the agent yourself and want the loop, tools and memory handled by a library
Agent Squad	An open-source multi-agent framework, once AWS Multi-Agent Orchestrator, now maintained outside awslabs: a classifier routes to a specialist and shared context follows the conversation	Work splits into distinct specialisms and one of them has to answer each turn
Amazon Bedrock AgentCore	A framework-agnostic agent platform of separable services: runtime, memory, gateway, identity, policy and observability	An agent you already wrote needs to run in production
Model Context Protocol (MCP)	The open protocol between an agent and its tools	You want one tool surface that several agents and several frameworks can all read
Kiro	An agentic development environment across an IDE, a CLI and the web, built for spec-driven development	Work should start from a specification the agent plans against rather than a blank completion
Amazon Q Developer	An assistant for code and your AWS account. The console, documentation and chat-app surfaces continue; the IDE plugins and Pro subscriptions closed to new signups in May 2026 and reach end of support on 30 April 2027, with Kiro as the successor	An engineer on an existing subscription needs help writing, reviewing or explaining code
Amazon Quick	A finished, managed assistant over enterprise data, enforcing each asker’s permissions, with Quick Flows and Quick Automate for automation, Quick Index for grounding and Quick Sight for BI	Staff need answers from internal systems without anyone building a product
Amazon Q Business	The enterprise assistant Quick superseded. Closed to new customers, with AWS pointing existing applications at Amazon Quick	You see the old name and need to recognise what replaced it

The three finished products are set against the platform in Kiro, Amazon Quick, or Bedrock.

Decision rules

• If traffic is spiky and the compute layer should cost nothing between bursts, then invoke Bedrock from Lambda on demand.
• If volume is steady and high, or the model has been customised in Bedrock, then reserve Provisioned Throughput in model units.
• If nobody is waiting and the pile is large, then submit a batch inference job against an S3 manifest.
• If the model is not a Bedrock FM, then it belongs on a SageMaker AI endpoint or a container you run.
• If a container serves the model, then size it for weights plus KV cache at your concurrency, and treat model load time as part of the scale-out latency.
• If a human is watching the answer appear, then stream it and pick a transport that forwards chunks.
• If the completion can run past thirty seconds, then get it off the request path: acknowledge, queue, and deliver the result out of band.
• If a REST API must stream, then set the proxy integration’s response transfer mode to STREAM, and give up endpoint caching and VTL response mapping.
• If an HTTP API must stream, then move to a WebSocket API or to Lambda response streaming through a function URL, and accept that usage plans do not follow you.
• If an answer must be current to the second, then call the source synchronously and put a usage plan in front of it.
• If seconds of lag are acceptable and more than one consumer may appear, then publish to EventBridge and buffer with SQS.
• If the source is fragile and cannot take per-question traffic, then sync a copy into S3 on a schedule and index that.
• If the source can push but cannot be polled, then take a webhook: verify the signature, write to SQS, and return 200 before doing slow work.
• If the data cannot leave a specific site, then put the storage and the redaction step on Outposts or at a Wavelength edge, and send only sanitised text to the Region.
• If several teams share one account’s quota, then a gateway with usage plans is the only place a per-team ceiling can be enforced.
• If a call fails with throttling, then back off with jitter before anything else. If it keeps failing, the workload is over quota and needs capacity, not retries.
• If a dependency is failing every call, then open a circuit breaker so requests fail fast instead of queueing behind a dead service.
• If prompts vary in difficulty within one model family, then point the request at an Intelligent Prompt Routing router, set the response quality difference, and check the per-request routing charge against what the cheaper model saves.
• If the easy majority is large and a reliable weakness signal exists, then cascade. If it does not, route on a rule you can read.
• If a release changes a prompt, a guardrail or a model id, then it goes through the same pipeline as code, with an evaluation gate before the deployment gate.

Traps

• A synchronous call behind API Gateway is capped by the integration timeout, not by the model. A completion that grew from twenty seconds to thirty-five starts returning 504 without anything in the model changing.
• Streaming does not make generation faster. It moves the first token forward; total time is the same, so a slow model is still slow.
• A proxy integration buffers the whole response until you set its transfer mode to STREAM. Calling ConverseStream behind a buffered one gets you streaming to the Lambda and a single blob to the browser.
• Lambda’s fifteen-minute ceiling is per invocation, not per workflow. Long chains split across steps. Managed Instances lift the ceiling to ninety minutes for asynchronous and event-source invocations, which does nothing for a client holding a connection open.
• Provisioned Throughput bills by the hour whether traffic arrives or not, so reserved capacity that sits idle overnight is still charged.
• Batch inference is cheap because nobody is waiting. It is never the answer when a user is watching a spinner, and it supports neither tool calling nor structured output.
• SQS visibility timeout has to exceed worst-case model latency. Otherwise a second worker picks up a document the first is still processing, and you pay twice for a duplicate answer.
• A webhook handler that is not idempotent on the provider’s delivery id will apply the same event twice, because deliveries retry and arrive out of order.
• Retries without backoff make throttling worse. Synchronised retries from many callers arrive as one spike against the same ceiling.
• A cross-Region inference profile raises throughput and changes where inference happens. Check the residency rules before enabling it.
• A cascade that escalates most requests costs more than sending everything to the capable model. Measure the escalation rate before shipping it.
• Intelligent Prompt Routing picks between two models in one family, not across arbitrary models, and adds USD$1 per 1,000 requests. On a cheap family that surcharge can outweigh the saving.
• An ensemble multiplies token spend by the number of models. It raises reliability on high-stakes answers and wastes money on everything else.
• Reserved concurrency on the worker is what keeps Lambda from scaling straight past the Bedrock quota. Without it, the fan-out that fixed your backlog creates the throttling.
• Amazon Q Developer and Amazon Quick sound alike and do different jobs. One helps with code and your AWS account, the other answers from company data with citations.

Say it in one line

1. Lambda on demand for spiky callers, Bedrock on-demand as the default, Provisioned Throughput for steady high volume or a customised model, batch for bulk with nobody waiting, SageMaker AI endpoints and containers for models Bedrock does not serve.
2. An LLM in a container is sized by weights plus KV cache, GPU utilisation and token throughput, and Fargate cannot supply the GPU.
3. Synchronous holds a connection and inherits every timeout. Streaming fixes the first-token feel. Asynchronous takes the work off the request path, and batch halves the token price and gives up immediacy.
4. A REST proxy integration streams only in STREAM transfer mode; otherwise use a WebSocket API or a Lambda function URL.
5. AppFlow is SaaS records, DataSync is file and object stores including on-premises NFS and SMB, Transfer Family is an SFTP, FTPS, FTP or AS2 drop into S3 or EFS.
6. Outposts and Wavelength hold data and pre-processing on site; Bedrock runs in the Region, so only sanitised text crosses the line.
7. A gateway in front of Bedrock is where per-team rate limits and mandatory guardrails become compulsory rather than optional.
8. Backoff with jitter for transient throttles, usage plans for noisy callers, circuit breakers for dead dependencies, fallback models for scarcity, cross-Region profiles for regional pressure, X-Ray to see which one fired.
9. Retries add no capacity. A workload structurally over quota needs a quota increase, Provisioned Throughput, or queued and deferred work.
10. Routing runs from static configuration through content-based rules and Intelligent Prompt Routing to cascades and ensembles, and the cost climbs with the number of models each request touches.
11. Prompts, guardrails and model ids ship through CodePipeline, CodeBuild, CodeDeploy and CDK like any other source, with an evaluation gate ahead of the deployment gate.
12. Strands builds the agent, Agent Squad coordinates several, AgentCore runs them in production, MCP is the protocol between agent and tools.
13. Kiro is spec-driven development and the successor to the Q Developer IDE plugins; Amazon Quick is the finished assistant over enterprise data that superseded Amazon Q Business.