Running Agents in Production With Bedrock AgentCore

The situation

The team has an agent that works. It was built on an open-source agent framework rather than the fully-managed Bedrock Agents experience, because the developers wanted direct control over the reasoning loop, the prompt structure, and which model answers each step. On a laptop it does everything asked of it: reasons, calls a couple of internal tools, and holds a conversation.

Now it has to serve subscribers. That changes the questions entirely. Two subscribers must never share a session or see each other’s context, so each run needs genuine isolation. A conversation that drops and reconnects should pick up where it left off, and a returning subscriber should not have to re-explain preferences the agent already learned, so there is short-term memory within a session and long-term memory across sessions. The agent needs to call internal APIs and a couple of third-party systems on the subscriber’s behalf, which means credentials, delegated access, and a way to not hand the model a standing key to everything. When a run misbehaves, someone has to be able to trace what the agent did, step by step, and see where it went wrong. And the whole thing has to scale from ten conversations to ten thousand without the team standing up and babysitting servers.

The team can build all of that themselves on containers and databases they operate, adopt the fully-managed Bedrock Agents path and give up the custom framework, or take the operational pieces from Bedrock AgentCore and keep the agent code they already have. Each choice moves the managed line to a different place.

What actually matters

The first thing to name is who owns the reasoning loop. The fully-managed Bedrock Agents experience owns it for you: AWS orchestrates the reason-act-observe cycle, and you supply instructions, tools, and knowledge. That is the least code and the least control. Bring-your-own-framework inverts it: your code, or your chosen open-source framework, runs the loop, and you decide the prompt structure, the tool-calling contract, and the model per step. AgentCore is built for that second world. It does not want to own your agent’s reasoning; it wants to run, remember, connect, secure, and observe an agent whose logic is yours.

The second is session isolation and scale. Multi-tenant agent traffic has a hard requirement that one subscriber’s execution cannot touch another’s, and a soft requirement that it scale without paged-out operators. A serverless agent runtime that runs each session in its own isolated execution context answers both: sessions do not share state, and capacity follows load without servers to manage. Building that yourself means containers, an isolation model you can defend, and autoscaling you operate. This is usually the piece that pushes a team off self-hosting first.

The third is memory, and it is two problems, not one. Short-term memory keeps the thread of a single conversation coherent across turns and reconnects. Long-term memory carries facts, preferences, and summaries across separate sessions so a returning subscriber is recognised. A managed memory capability gives you both without standing up and tuning your own stores; rolling it yourself means a datastore, a retrieval strategy, and a retention policy you design and maintain.

The fourth is identity and tools, which are entangled. An agent is only as useful as the systems it can reach, and only as safe as the access it is granted. Two capabilities sit here. A gateway turns existing APIs and Lambda functions into tools the agent can call through a consistent tool interface aligned with MCP-style conventions, so you expose what you already have rather than rewriting it as agent actions. An identity capability handles delegated access: letting the agent act against AWS services and third-party systems with scoped, brokered credentials rather than a broad standing key baked into the code. The safety story lives here, so it is worth weighing on its own.

The fifth is observability, because an agent you cannot trace is an agent you cannot operate. Non-deterministic reasoning, tool calls that sometimes fail, and multi-step runs mean the difference between a debuggable system and an opaque one is whether you can see the trace: which steps ran, what each tool returned, where latency and cost went, and where a run broke. Managed observability for agent runs gives you that surface; without it you instrument everything yourself.

Underneath all of it: you take the pieces you need, not the whole set. AgentCore’s capabilities are usable independently. A team might want only the runtime and observability and keep its own memory; another might adopt memory and identity around an agent that already runs elsewhere. The decision is rarely all-or-nothing.

What we’ll filter on

1. Do you need to bring your own agent framework and choose your own model, or is the fully-managed, AWS-orchestrated experience enough?
2. Do you need managed memory (short-term within a session, long-term across sessions) and managed identity for delegated access?
3. What are the production requirements for session isolation, secure execution, and scaling under real traffic?
4. Do you need built-in traceability, metrics, and debugging for non-deterministic agent runs?
5. How much of the orchestration and operational surface do you want AWS to own versus keep in your own hands?

The AgentCore landscape

Bedrock AgentCore is a set of operational building blocks for deploying and running AI agents securely at scale. It is framework-agnostic, so it works with agents built on various open-source agent frameworks, and model-agnostic, so the model behind the agent is your choice rather than a fixed one. The pieces are usable together or independently.

1. A serverless agent runtime. Runs your agent code in a managed, serverless environment with per-session isolation, so each subscriber’s run executes in its own context without sharing state with another’s. It handles scaling with load and removes the servers you would otherwise operate. This is the home for a bring-your-own-framework agent that needs to run in production without you managing the compute or the isolation model.

2. Memory. A managed capability for both short-term memory, keeping a single conversation coherent across turns and reconnects, and long-term memory, carrying facts, preferences, and summaries across separate sessions so a returning subscriber is recognised. It removes the datastore, retrieval strategy, and retention policy you would otherwise design and run yourself.

3. A gateway that turns tools on. Takes existing APIs and Lambda functions and exposes them as tools the agent can call through a consistent tool interface aligned with MCP-style conventions. The point is reuse: you make what you already own available to the agent without rewriting it as bespoke agent actions, and the agent sees a uniform way to discover and call those tools.

4. Identity for delegated access. Lets the agent act against AWS services and third-party systems with scoped, brokered credentials rather than a broad standing key embedded in the code. It is the capability that answers “how does this agent reach that system safely”, and it is where the least-privilege story for an agent lives.

5. Observability. Tracing, metrics, and debugging for agent runs: which steps executed, what each tool returned, where time and cost went, and where a run failed. It turns a non-deterministic, multi-step agent from an opaque process into one you can inspect and operate.

6. Built-in tools. Ready-made capabilities an agent commonly needs, including a sandboxed code interpreter for running generated code safely and a browser for reaching the web, so you are not building and securing those primitives from scratch.

For contrast, two things sit on either side of AgentCore. The fully-managed Bedrock Agents experience is the simpler path where AWS orchestrates the reasoning loop and you give up the custom framework; that shape, and how it composes into multi-agent setups, is covered in orchestrating multiple Bedrock agents. A fully self-hosted stack is the other end: your own containers, datastores, credential broker, and instrumentation, with total control and total operational burden.

Side by side

Capability or need	Fully-managed Bedrock Agents	Bedrock AgentCore	Fully self-hosted
Bring your own framework	✗ (AWS orchestrates)	✓	✓
Choose your own model	Partial (managed set)	✓	✓
Serverless runtime with session isolation	✓ (managed for you)	✓	✗ (you build it)
Managed short and long-term memory	✓	✓	✗ (you build it)
Expose existing APIs and Lambdas as tools	✓ (action groups)	✓ (gateway)	✗ (you build it)
Delegated, scoped identity	✓	✓	✗ (you build it)
Built-in tracing and debugging	✓	✓	✗ (you build it)
Sandboxed code interpreter and browser	Partial	✓	✗ (you build it)
You own the reasoning loop	✗	✓	✓
Pick capabilities independently	✗ (one experience)	✓	N/A (all yours)

Reading it for this situation, an agent already built on an open-source framework that has to go multi-tenant, the fully-managed experience is off the table unless the team will give up the custom loop, and full self-hosting means rebuilding isolation, memory, identity, and tracing from nothing. AgentCore is the middle: keep the agent code, take the operational pieces that are hard to get right.

The capabilities around the agent

The picks in depth

Start with the runtime and observability. For a bring-your-own-framework agent going multi-tenant, these two are the pieces that are both hardest to build well and most dangerous to get wrong. The serverless runtime runs each session in its own isolated context, which is the guarantee that one subscriber cannot see another’s conversation, and it scales with traffic so there is no server fleet to size. Observability turns the run from a black box into a trace: you can see which steps executed, what each tool returned, and where a failure or a latency spike came from. An agent you cannot trace is an agent you cannot operate, so this pair is usually the first reason a team stops self-hosting.

Add memory when statelessness starts to hurt. Short-term memory is what keeps a single conversation coherent when a connection drops and reconnects mid-task; without it, every reconnect is a fresh, forgetful start. Long-term memory is what lets a returning subscriber skip re-explaining preferences the agent already learned, carrying facts and summaries across separate sessions. Building both means choosing a datastore, a retrieval strategy, and a retention policy and then keeping them tuned; the managed capability is worth it precisely when that maintenance is not where the team wants to spend its time. You can also keep your own memory and take only the other pieces, so this is an opt-in, not a requirement.

Take identity and the gateway together, because access and tools are the same conversation. The gateway lets you expose the internal APIs and Lambda functions you already have as tools the agent can call through a consistent, MCP-aligned interface, so you reuse rather than rewrite. Identity is what makes those calls safe: scoped, brokered credentials for the agent to act against AWS services and third-party systems on the subscriber’s behalf, instead of a broad standing key baked into the code. The least-privilege story for an agent lives in this pair, and for anything that touches subscriber data or moves money it is the part to get right first. The built-in tools, a sandboxed code interpreter and a browser, sit alongside as ready-made capabilities you would otherwise have to build and secure yourself.

Weigh it against the two neighbours. The fully-managed Bedrock Agents experience is the right answer when the team does not need its own framework and is happy to let AWS orchestrate the loop; it is less code and less to own, at the price of the custom control the team built the agent to have. Full self-hosting is the right answer only when a requirement genuinely cannot be met by the managed pieces, because it means rebuilding isolation, memory, identity, tracing, and the sandboxed primitives yourself, and then operating all of them. AgentCore is the middle path: keep the agent you have, and adopt the operational capabilities that are expensive to build and risky to get wrong, one at a time as production demands them.

A worked example: the prototype goes multi-tenant

The team’s agent runs on an open-source framework and answers subscriber questions well in a single-user prototype. The move to production happens in the order the pain arrives.

First, isolation and scale. Ten thousand subscribers cannot share one process, and the team does not want a server fleet. The agent code is deployed onto the serverless runtime, which runs each session in its own isolated context and scales with load. Nothing about the reasoning loop changes; only where it runs does. At the same time, observability is switched on, so the first production incident is debuggable rather than a guess: the trace shows a third-party tool timing out on a specific step.

Next, memory. Support conversations drop and reconnect over flaky mobile connections, and subscribers were re-explaining their situation every time. Short-term memory keeps each conversation coherent across reconnects. Then returning subscribers start noticing the agent forgets preferences between contacts, so long-term memory is added to carry those facts across sessions. The team did not stand up a datastore for either.

Then tools and access. The agent needs to check billing and update a delivery preference, both behind internal APIs the team already runs. The gateway exposes those APIs as tools without rewriting them, and identity issues the agent scoped credentials to call them on the subscriber’s behalf, so there is no broad standing key in the code and the billing tool’s access is limited to what it needs. A later feature that runs a small calculation uses the built-in sandboxed code interpreter rather than a bespoke execution service.

The framework and the model were the team’s choices throughout. What changed on the way to production was the operational layer around the agent, taken a piece at a time as each requirement bit, and none of it was rebuilt from scratch.

What’s worth remembering

1. AgentCore is the operational layer for agents you build yourself: framework-agnostic and model-agnostic, it runs, remembers, connects, secures, and observes an agent whose reasoning loop stays yours.
2. The decision axis is where the managed line sits: AWS-orchestrated fully-managed agents on one side, a fully self-hosted stack on the other, AgentCore in the middle keeping your framework while taking the hard operational pieces.
3. The runtime is serverless with per-session isolation, so multi-tenant traffic cannot bleed across subscribers and capacity follows load without a server fleet to operate.
4. Memory is two problems: short-term keeps one conversation coherent across turns and reconnects, long-term carries facts and preferences across separate sessions.
5. The gateway turns existing APIs and Lambda functions into agent-callable tools through a consistent, MCP-aligned interface, so you reuse what you own rather than rewriting it.
6. Identity handles delegated access with scoped, brokered credentials, keeping a broad standing key out of the agent and putting least privilege where the agent reaches other systems.
7. Observability gives you the trace, metrics, and debugging that make a non-deterministic, multi-step agent operable instead of opaque.
8. Built-in tools, a sandboxed code interpreter and a browser, save you building and securing those primitives yourself.
9. The capabilities are independent; adopt the runtime and observability first, then memory, identity, and the gateway as production demands them, and keep your own where you prefer.
10. Choose the fully-managed experience when you do not need your own framework, self-hosting only when a requirement the managed pieces cannot meet forces it, and AgentCore when you want production-grade operations around an agent that is already yours.