The situation
The team has an agent that works. It was built on an open-source agent framework rather than declared as configuration, because the developers wanted direct control over the reasoning loop, the prompt structure, and which model answers each step. On a laptop it does everything asked of it: reasons, calls a couple of internal tools, and holds a conversation.
Now it has to serve subscribers. That changes the questions entirely. Two subscribers must never share a session or see each other’s context, so each run needs genuine isolation. A conversation that drops and reconnects should pick up where it left off, and a returning subscriber should not have to re-explain preferences the agent already learned, so there is short-term memory within a session and long-term memory across sessions. The agent needs to call internal APIs and a couple of third-party systems on the subscriber’s behalf, which means credentials, delegated access, and a way to not hand the model a standing key to everything. When a run misbehaves, someone has to be able to trace what the agent did, step by step, and see where it went wrong. And the whole thing has to scale from ten conversations to ten thousand without the team standing up and babysitting servers.
The team can build all of that themselves on containers and databases they operate, take the operational pieces from Bedrock AgentCore and keep the agent code they already have, or give up the custom loop entirely and declare the agent to AgentCore’s managed harness as a model, a set of tools, and some instructions. Each shape moves the managed line to a different place, and the loop is the thing being traded.
What actually matters
The first thing to name is who owns the reasoning loop. The managed harness owns it for you: you declare a model, a system prompt, and a set of tools, and AgentCore orchestrates the reason-act-observe cycle. That is the least code and the least control, and for a great many agents it is enough. Bringing your own framework inverts it: your code runs the loop, and you decide the prompt structure, the tool-calling contract, and the model per step. AgentCore serves both, which is what makes this a real decision rather than a platform choice. The same runtime, memory, gateway, identity, and observability sit underneath either one; what changes is whether the loop is yours.
The second is session isolation and scale. Multi-tenant agent traffic has a hard requirement that one subscriber’s execution cannot touch another’s, and a soft requirement that it scale without paged-out operators. A serverless agent runtime that runs each session in its own isolated execution context answers both: sessions do not share state, and capacity follows load without servers to manage. Building that yourself means containers, an isolation model you can defend, and autoscaling you operate. This is usually the piece that pushes a team off self-hosting first.
The third is memory, and it is two problems, not one. Short-term memory keeps the thread of a single conversation coherent across turns and reconnects. Long-term memory carries facts, preferences, and summaries across separate sessions so a returning subscriber is recognised. A managed memory capability gives you both without standing up and tuning your own stores; rolling it yourself means a datastore, a retrieval strategy, and a retention policy you design and maintain.
The fourth is identity and tools, which are entangled. An agent is only as useful as the systems it can reach, and only as safe as the access it is granted. Two capabilities sit here. A gateway turns existing APIs and Lambda functions into tools the agent can call through a consistent tool interface aligned with MCP-style conventions, so you expose what you already have rather than rewriting it as agent actions. An identity capability handles delegated access: letting the agent act against AWS services and third-party systems with scoped, brokered credentials rather than a broad standing key baked into the code. The safety story lives here, so it is worth weighing on its own.
The fifth is observability, because an agent you cannot trace is an agent you cannot operate. Non-deterministic reasoning, tool calls that sometimes fail, and multi-step runs mean the difference between a debuggable system and an opaque one is whether you can see the trace: which steps ran, what each tool returned, where latency and cost went, and where a run broke. Managed observability for agent runs gives you that surface; without it you instrument everything yourself.
Underneath all of it: you take the pieces you need, not the whole set. AgentCore’s capabilities are usable independently. A team might want only the runtime and observability and keep its own memory; another might adopt memory and identity around an agent that already runs elsewhere. The decision is rarely all-or-nothing.
What we’ll filter on
- Do you need to own the reasoning loop, with custom orchestration and stage-specific prompt control, or is a declared model, prompt, and tool list enough?
- Do you need managed memory (short-term within a session, long-term across sessions) and managed identity for delegated access?
- What are the production requirements for session isolation, secure execution, and scaling under real traffic?
- Do you need built-in traceability, metrics, and debugging for non-deterministic agent runs?
- How much of the orchestration and operational surface do you want AWS to own versus keep in your own hands?
The landscape
Bedrock AgentCore is a set of operational building blocks for deploying and running AI agents securely at scale. It is framework-agnostic, so it works with agents built on various open-source agent frameworks, and model-agnostic, so the model behind the agent is your choice rather than a fixed one. The pieces are usable together or independently.
A serverless agent runtime. Runs your agent code in a managed, serverless environment with per-session isolation, so each subscriber’s run executes in its own context without sharing state with another’s. It handles scaling with load and removes the servers you would otherwise operate. This is the home for a bring-your-own-framework agent that needs to run in production without you managing the compute or the isolation model.
Memory. A managed capability for both short-term memory, keeping a single conversation coherent across turns and reconnects, and long-term memory, carrying facts, preferences, and summaries across separate sessions so a returning subscriber is recognised. It removes the datastore, retrieval strategy, and retention policy you would otherwise design and run yourself.
A gateway for the tools you already have. Takes existing APIs and Lambda functions and publishes them as tools the agent can call through a consistent interface aligned with MCP-style conventions. You expose what you already own rather than rewriting it as bespoke agent actions, and the agent gets one uniform way to discover and call any of it.
Identity for delegated access. Lets the agent act against AWS services and third-party systems with scoped, brokered credentials rather than a broad standing key embedded in the code. It is the capability that answers “how does this agent reach that system safely”, and it is where the least-privilege story for an agent lives.
Observability. Tracing, metrics, and debugging for agent runs: which steps executed, what each tool returned, where time and cost went, and where a run failed. It turns a non-deterministic, multi-step agent from an opaque process into one you can inspect and operate.
Built-in tools. Ready-made capabilities an agent commonly needs, including a sandboxed code interpreter for running generated code safely and a browser for reaching the web, so you are not building and securing those primitives from scratch.
Two shapes sit on either side of a code-defined agent. The managed harness is the simpler path: declare the model, the instructions, and the tools, and AgentCore runs the loop, with memory on by default and a model you can override per invocation without redeploying. You give up custom orchestration, stage-specific prompt overrides, and straightforward multi-agent routing, and you get a working agent from a few CLI commands. How more than one agent composes is covered in orchestrating multiple agents. A fully self-hosted stack is the other end: your own containers, datastores, credential broker, and instrumentation, with total control and total operational burden.
Evaluation
Side by side
| Capability or need | AgentCore harness | Your loop on AgentCore | Fully self-hosted |
|---|---|---|---|
| You own the reasoning loop | ✗ (declared, not written) | ✓ | ✓ |
| Bring your own framework | ✗ | ✓ (Strands, LangChain, any) | ✓ |
| Choose your own model | ✓ (overridable per invocation) | ✓ | ✓ |
| Custom orchestration and multi-agent routing | ✗ (agent-as-tool only) | ✓ | ✓ |
| Prompt control at specific stages | ✗ (one system prompt) | ✓ | ✓ |
| Serverless runtime with session isolation | ✓ | ✓ | ✗ (you build it) |
| Managed short and long-term memory | ✓ (on by default) | ✓ | ✗ (you build it) |
| Expose existing APIs and Lambdas as tools | ✓ (gateway) | ✓ (gateway) | ✗ (you build it) |
| Delegated, scoped identity | ✓ | ✓ | ✗ (you build it) |
| Tracing and debugging | ✓ | ✓ (instrument with ADOT) | ✗ (you build it) |
| Sandboxed code interpreter and browser | ✓ | ✓ | ✗ (you build it) |
| Pick capabilities independently | Partial (bundled) | ✓ | N/A (all yours) |
Reading it for this situation, an agent already built on an open-source framework that has to go multi-tenant: the harness is out because adopting it means throwing away the loop the team deliberately wrote, and full self-hosting means rebuilding isolation, memory, identity, and tracing from nothing. The middle column is the answer. Keep the agent code, take the operational pieces that are hard to get right. Worth noticing how few rows separate the first two columns, though, because for a team without that existing loop the harness gets almost everything the runtime does for a fraction of the code.
The capabilities around the agent
The solution
Start with the runtime and observability. For a bring-your-own-framework agent going multi-tenant, these two are the pieces that are both hardest to build well and most dangerous to get wrong. The serverless runtime runs each session in its own isolated context, which is the guarantee that one subscriber cannot see another’s conversation, and it scales with traffic so there is no server fleet to size. Observability turns the run from a black box into a trace: you can see which steps executed, what each tool returned, and where a failure or a latency spike came from. An agent you cannot trace is an agent you cannot operate, so this pair is usually the first reason a team stops self-hosting.
Add memory when statelessness starts to hurt. Short-term memory is what keeps a single conversation coherent when a connection drops and reconnects mid-task; without it, every reconnect is a fresh, forgetful start. Long-term memory is what lets a returning subscriber skip re-explaining preferences the agent already learned, carrying facts and summaries across separate sessions. Building both means choosing a datastore, a retrieval strategy, and a retention policy and then keeping them tuned; the managed capability is worth it precisely when that maintenance is not where the team wants to spend its time. You can also keep your own memory and take only the other pieces, so this is an opt-in, not a requirement.
Take identity and the gateway together, because access and tools are the same conversation. The gateway lets you expose the internal APIs and Lambda functions you already have as tools the agent can call through a consistent, MCP-aligned interface, so you reuse rather than rewrite. Identity is what makes those calls safe: scoped, brokered credentials for the agent to act against AWS services and third-party systems on the subscriber’s behalf, instead of a broad standing key baked into the code. The least-privilege story for an agent lives in this pair, and for anything that touches subscriber data or moves money it is the part to get right first. The built-in tools, a sandboxed code interpreter and a browser, sit alongside as ready-made capabilities you would otherwise have to build and secure yourself.
Weigh it against the two neighbours. The harness is the right answer when nobody needs to own the loop: it is less code and less to own, at the price of the custom control this team built their agent to have, and it is where a fresh build should start unless something specific rules it out. Full self-hosting is the right answer only when a requirement genuinely cannot be met by the managed pieces, because it means rebuilding isolation, memory, identity, tracing, and the sandboxed primitives yourself, and then operating all of them. AgentCore is the middle path: keep the agent you have, and adopt the operational capabilities that are expensive to build and risky to get wrong, one at a time as production demands them.
Worked example
The team’s agent runs on an open-source framework and answers subscriber questions well in a single-user prototype. The move to production happens in the order the pain arrives.
First, isolation and scale. Ten thousand subscribers cannot share one process, and the team does not want a server fleet. The agent code is deployed onto the serverless runtime, which runs each session in its own isolated context and scales with load. Nothing about the reasoning loop changes; only where it runs does. At the same time, observability is switched on, so the first production incident is debuggable rather than a guess: the trace shows a third-party tool timing out on a specific step.
Next, memory. Support conversations drop and reconnect over flaky mobile connections, and subscribers were re-explaining their situation every time. Short-term memory keeps each conversation coherent across reconnects. Then returning subscribers start noticing the agent forgets preferences between contacts, so long-term memory is added to carry those facts across sessions. The team did not stand up a datastore for either.
Then tools and access. The agent needs to check billing and update a delivery preference, both behind internal APIs the team already runs. The gateway exposes those APIs as tools without rewriting them, and identity issues the agent scoped credentials to call them on the subscriber’s behalf, so there is no broad standing key in the code and the billing tool’s access is limited to what it needs. A later feature that runs a small calculation uses the built-in sandboxed code interpreter rather than a bespoke execution service.
The framework and the model were the team’s choices throughout. What changed on the way to production was the operational layer around the agent, taken a piece at a time as each requirement bit, and none of it was rebuilt from scratch.
What’s worth remembering
- AgentCore is the operational layer for agents you build yourself: framework-agnostic and model-agnostic, it runs, remembers, connects, secures, and observes an agent whose reasoning loop stays yours.
- The decision axis is where the managed line sits: a declared agent on the harness at one end, a fully self-hosted stack at the other, and a loop of your own on the AgentCore runtime in between.
- The runtime is serverless with per-session isolation, so multi-tenant traffic cannot bleed across subscribers and capacity follows load without a server fleet to operate.
- The capabilities are independent; adopt the runtime and observability first, then memory, identity, and the gateway as production demands them, and keep your own where you prefer.
- Start on the harness unless you have a named reason to own the loop, drop to a code-defined agent on the runtime when you do, and self-host only when a requirement the managed pieces cannot meet forces it.