The situation
The team has an agent that works. It was built on an open-source agent framework rather than declared as configuration, because the developers wanted direct control over the reasoning loop, the prompt structure, and which model answers each step. On a laptop it does everything asked of it: reasons, calls a couple of internal tools, and holds a conversation.
Now it has to serve subscribers. That changes the questions entirely. Two subscribers must never share a session or see each other’s context, so each run needs genuine isolation. A conversation that drops and reconnects should pick up where it left off, and a returning subscriber should not have to re-explain preferences the agent already learned, so there is short-term memory within a session and long-term memory across sessions. The agent needs to call internal APIs and a couple of third-party systems on the subscriber’s behalf, which means credentials, delegated access, and a way to not hand the model a standing key to everything. When a run goes wrong, someone has to be able to trace what the agent did, step by step, and see where it went wrong. And the whole thing has to scale from ten conversations to ten thousand without the team standing up and babysitting servers.
The team can build all of that themselves on containers and databases they operate, take the operational pieces from Bedrock AgentCore and keep the agent code they already have, or give up the custom loop entirely and declare the agent to AgentCore’s managed harness as a model, a set of tools, and some instructions. Each shape moves the managed line to a different place, and the loop is the thing being traded.
What actually matters
The first thing to name is who owns the reasoning loop. AgentCore’s harness owns it for you: you declare a model, a system prompt, tools, memory, and execution limits as configuration, and the harness runs the reason-act-observe cycle. That is the least code and the least control, and for a great many agents it is enough. Bringing your own framework inverts it: your code runs the loop, and you decide the prompt structure, the tool-calling contract, and the model per step. The same memory, gateway, identity, and observability sit underneath either one, and the harness itself runs inside the AgentCore runtime. So this settles where your code stops; AgentCore sits underneath either way.
The second is session isolation and scale. Multi-tenant agent traffic has a hard requirement that one subscriber’s execution cannot touch another’s, and a second requirement that it scale without a team on call for capacity. A serverless agent runtime that gives each session a dedicated microVM, with isolated CPU, memory, and filesystem, answers both. Sessions cannot read each other’s state, the microVM is terminated and its memory sanitised when the session ends, and capacity follows load without servers to manage. Building that yourself means containers, an isolation model you can defend, and autoscaling you operate. This is usually the piece that pushes a team off self-hosting first.
The third is memory, and it is two problems, not one. Short-term memory keeps the thread of a single conversation coherent across turns and reconnects. Long-term memory carries facts, preferences, and summaries across separate sessions so a returning subscriber is recognised. A managed memory capability gives you both without standing up and tuning your own stores; rolling it yourself means a datastore, a retrieval strategy, and a retention policy you design and maintain.
The fourth is identity and tools, which are entangled. An agent is only as useful as the systems it can reach, and only as safe as the access it is granted. Two capabilities sit here. A gateway converts existing APIs and Lambda functions into Model Context Protocol tools the agent can call, so you expose what you already have rather than rewriting it as agent actions. An identity capability handles delegated access: letting the agent act against AWS services and third-party systems with scoped, brokered credentials rather than a broad standing key baked into the code. Access control is worth weighing on its own.
The fifth is observability, because an agent you cannot trace is an agent you cannot operate. Non-deterministic reasoning, tool calls that sometimes fail, and multi-step runs mean the difference between a debuggable system and an opaque one is whether you can see the trace: which steps ran, what each tool returned, where latency and cost went, and where a run broke. AgentCore emits built-in metrics for runtime, gateway, and memory resources by default, and full trace visualisation needs CloudWatch Transaction Search turned on once per account plus the ADOT SDK in your agent code. Without any of it you build the whole surface yourself.
Underneath all of it: you take the pieces you need, not the whole set. AgentCore’s capabilities are usable independently. A team might want only the runtime and observability and keep its own memory; another might adopt memory and identity around an agent that already runs elsewhere. The decision is rarely all-or-nothing.
What we’ll filter on
- Do you need to own the reasoning loop, with control flow of your own, or is a declared model, prompt, and tool list enough?
- Do you need managed memory (short-term within a session, long-term across sessions) and managed identity for delegated access?
- What are the production requirements for session isolation, secure execution, and scaling under real traffic?
- Do you need built-in traceability, metrics, and debugging for non-deterministic agent runs?
- How much of the orchestration and operational surface do you want AWS to own versus keep in your own hands?
The landscape
Bedrock AgentCore is a set of operational building blocks for deploying and running AI agents securely at scale. It is framework-agnostic, running agents built on CrewAI, LangGraph, LlamaIndex, Google ADK, the OpenAI Agents SDK, and Strands Agents among others, and model-agnostic, so the model behind the agent is your choice rather than a fixed one. The pieces are usable together or independently.
A serverless agent runtime. Runs your agent code in a managed environment with per-session isolation, each session in its own microVM, so a subscriber’s run never shares state with another’s. You wrap the code with the AgentCore SDK entrypoint, package it as an ARM64 container, push it to Amazon ECR, and deploy. Sessions run up to 8 hours on microVMs, or up to 14 days on Instances, a second compute type that runs on AWS-managed EC2 in your own account for long or GPU-backed work. Payloads go up to 100MB, and the runtime carries MCP and A2A traffic to other agents and tools.
Memory. A managed capability for both short-term memory, keeping a single conversation coherent across turns and reconnects, and long-term memory, carrying facts, preferences, and summaries across separate sessions so a returning subscriber is recognised. It removes the datastore, retrieval strategy, and retention policy you would otherwise design and run yourself.
A gateway for the tools you already have. Converts existing APIs, Lambda functions, and services into MCP-compatible tools, taking OpenAPI, Smithy, and Lambda as input types, and also fronts pre-existing MCP servers, other agents over A2A, and inference traffic across model providers. Semantic tool selection lets an agent search a large tool catalogue instead of carrying all of it in the prompt.
Identity for delegated access. Lets the agent act against AWS services and third-party systems with scoped, brokered credentials rather than a broad standing key embedded in the code. It answers how the agent reaches a given system safely, and it is where least privilege for an agent is enforced.
Observability. Tracing, metrics, and debugging for agent runs: which steps executed, what each tool returned, where time and token usage went, and where a run failed. Telemetry is emitted in OpenTelemetry format and lands in CloudWatch, with a generative AI observability dashboard over the runtime traces.
Policy. Deterministic rules, written in Cedar or generated from plain English, held in a policy engine attached to a gateway. Every tool call is intercepted and evaluated before it runs, so the boundary sits outside the agent’s code and cannot be talked around by a prompt. Evaluations and Optimization sit alongside it for scoring agent behaviour and tuning prompts and tool descriptions against real traces.
Built-in tools. Ready-made capabilities an agent commonly needs, including a sandboxed code interpreter for running generated code safely and a browser for reaching the web, so you are not building and securing those primitives from scratch.
Two shapes sit on either side of a code-defined agent. The harness is the simpler path: declare the model, the instructions, and the tools, and AgentCore runs the loop, which is powered by Strands Agents and hosted on the same runtime. Defaults are set at creation and overridden per invocation, so the model, system prompt, tools, and iteration and token limits all change without a redeploy. You can also switch model provider between turns of one session, with the conversation intact. What you give up is named in the docs: no choice of agent framework, no graph or workflow patterns outside the agent loop, no hooks, and no bidirectional streaming. Memory is a config field rather than an assumption. A harness created through the service API with the memory configuration omitted gets managed memory; the CLI creates one without memory unless you ask for it. When configuration stops being enough, you can export the harness to Strands code and run it on the runtime. How more than one agent composes is covered in orchestrating multiple agents. A fully self-hosted stack is the other end: your own containers, datastores, credential broker, and instrumentation, with total control and total operational burden.
Evaluation
Side by side
| Capability or need | AgentCore harness | Your loop on AgentCore | Fully self-hosted |
|---|---|---|---|
| You own the reasoning loop | ✗ (declared, not written) | ✓ | ✓ |
| Choice of agent framework | ✗ (Strands, fixed) | ✓ (CrewAI, LangGraph, any) | ✓ |
| Choose your own model | ✓ (switch provider mid-session) | ✓ | ✓ |
| Graph and workflow patterns outside the loop | ✗ | ✓ | ✓ |
| Hooks and bidirectional streaming | ✗ | ✓ (your code) | ✓ |
| Per-session microVM isolation | ✓ | ✓ | ✗ (you build it) |
| Managed short and long-term memory | ✓ (config field) | ✓ (your code calls it) | ✗ (you build it) |
| Expose existing APIs and Lambdas as tools | ✓ (gateway) | ✓ (gateway) | ✗ (you build it) |
| Delegated, scoped identity | ✓ | ✓ (your code calls it) | ✗ (you build it) |
| Deterministic policy on every tool call | ✓ (gateway) | ✓ (gateway) | ✗ (you build it) |
| Tracing and debugging | ✓ | ✓ (instrument with ADOT) | ✗ (you build it) |
| Sandboxed code interpreter and browser | ✓ | ✓ | ✗ (you build it) |
Reading it for this situation, an agent already built on an open-source framework that has to go multi-tenant: the harness is out because adopting it means throwing away the loop the team deliberately wrote, and full self-hosting means rebuilding isolation, memory, identity, and tracing from nothing. The middle column is the answer. Keep the agent code, take the operational pieces that are hard to get right. Notice how few rows separate the first two columns. For a team without an existing loop, the harness reaches almost everything the runtime does from a handful of CLI commands, and the rows it loses are all about control flow.
The capabilities around the agent
The solution
Start with the runtime and observability. For a bring-your-own-framework agent going multi-tenant, these two are the pieces that are both hardest to build well and most dangerous to get wrong. The runtime gives each session its own microVM and terminates it afterwards, which is the guarantee that one subscriber cannot see another’s conversation, and it scales with traffic so there is no server fleet to size. Observability turns the run from a black box into a trace of which steps executed, what each tool returned, and where a failure or a latency spike came from. Two setup steps matter here: CloudWatch Transaction Search is a one-time account-level switch, and a bring-your-own-framework agent needs the ADOT SDK in its image, launched under opentelemetry-instrument, before its spans reach CloudWatch.
Add memory when statelessness starts to hurt. Short-term memory is what keeps a single conversation coherent when a connection drops and reconnects mid-task; without it, every reconnect is a fresh, forgetful start. Long-term memory is what lets a returning subscriber skip re-explaining preferences the agent already learned, carrying facts and summaries across separate sessions. Long-term memory runs on configurable strategies, semantic extraction, summarisation, user preference, and episodic, and the agent queries them and injects the results before it reasons. Building the same thing means choosing a datastore, a retrieval strategy, and a retention policy and then keeping them tuned. You can also keep your own memory store and take only the other pieces, so this is an opt-in rather than a requirement.
Take identity and the gateway together, because access and tools are settled at once. The gateway lets you expose the internal APIs and Lambda functions you already have as tools the agent can call through a consistent, MCP-compatible interface, so you reuse rather than rewrite. Identity is what makes those calls safe: scoped, brokered credentials for the agent to act against AWS services and third-party systems on the subscriber’s behalf, instead of a broad standing key baked into the code. Policy sits on the same gateway and evaluates every tool call against Cedar rules before it runs, which keeps the limit on what the agent may do outside the prompt and outside the agent’s code. For anything that touches subscriber data or moves money, get this trio right first. The built-in tools, a sandboxed code interpreter and a browser, sit alongside as ready-made capabilities you would otherwise have to build and secure yourself.
Weigh it against the two neighbours. The harness is the right answer when nobody needs to own the loop. It is less code to write and less to operate, and it gives up the framework choice and the control flow this team built their agent around. A fresh build should start there unless something specific rules it out, and the export path to Strands code means starting there is not a dead end. Full self-hosting is the right answer only when a requirement genuinely cannot be met by the managed pieces, because it means rebuilding isolation, memory, identity, tracing, and the sandboxed primitives yourself, then operating all of them. AgentCore is the middle path: keep the agent you have, and adopt the operational capabilities that are slow to build and risky to get wrong, one at a time as production demands them.
Worked example
The team’s agent runs on an open-source framework and answers subscriber questions well in a single-user prototype. The move to production happens in the order the pain arrives.
First, isolation and scale. Ten thousand subscribers cannot share one process, and the team does not want a server fleet. The agent is packaged as an ARM64 container, pushed to ECR, and deployed onto the runtime, which gives each session its own microVM and scales with load. Nothing about the reasoning loop changes; only where it runs does. The image picks up the ADOT SDK and the account has Transaction Search enabled, so the first production incident is debuggable rather than a guess: the trace shows a third-party tool timing out on a specific step.
Next, memory. Support conversations drop and reconnect over flaky mobile connections, and subscribers were re-explaining their situation every time. Short-term memory keeps each conversation coherent across reconnects. Then returning subscribers notice that nothing carries over between contacts, so long-term memory is added to hold those facts across sessions. The team did not stand up a datastore for either.
Then tools and access. The agent needs to check billing and update a delivery preference, both behind internal APIs the team already runs. The gateway publishes those APIs as MCP tools without rewriting them, and identity issues the agent scoped credentials to call them on the subscriber’s behalf, so there is no broad standing key in the code. A Cedar policy on the gateway limits the delivery-preference tool to the actor whose subscription it names, and that rule is evaluated before the call runs. A later feature that runs a small calculation uses the built-in sandboxed code interpreter rather than a bespoke execution service.
The framework and the model were the team’s choices throughout. What changed on the way to production was the operational layer around the agent, taken a piece at a time as each requirement arrived, and none of it was rebuilt from scratch.
What’s worth remembering
- AgentCore is the operational layer around an agent whose reasoning loop stays yours: framework-agnostic, model-agnostic, and made of pieces you adopt one at a time.
- The decision axis is where the managed line sits: a declared agent on the harness at one end, a fully self-hosted stack at the other, and a loop of your own on the AgentCore runtime in between.
- The runtime gives each session a dedicated microVM and destroys it afterwards, so multi-tenant traffic cannot bleed across subscribers; sessions run up to 8 hours there, or up to 14 days on Instances.
- The harness is configuration rather than code, powered by Strands and hosted on the runtime, and what it rules out is control flow: framework choice, graph and workflow patterns, hooks, bidirectional streaming.
- Gateway converts APIs, Lambda functions, and services into MCP tools; Identity brokers scoped credentials for them; Policy evaluates Cedar rules on every call before it runs.
- Built-in metrics arrive by default, but agent traces need CloudWatch Transaction Search enabled and the ADOT SDK in your own agent code.