How to Wire an LLM to Side-Effecting Actions with Bedrock AgentCore

The situation

Support engineering has a next-step list for the assistant that currently only answers questions. The new asks are actions:

• Look up a customer’s subscription by email or ID. Hits an internal subscriptions API.
• Pause or resume a subscription. Same API, different endpoint, side-effecting.
• Issue a refund for a specific charge. Hits the billing service; writes to the ledger.
• Send a confirmation email after any action. Hits the notifications service.

Four tools. Each lives behind an internal HTTPS API with OAuth2 client credentials. Each has a JSON schema. Each has a blast radius: the lookup is safe; the pause is reversible; the refund is money changing hands. The assistant has to pick the right tool, pass the correct arguments, handle errors, and stop and confirm with the user before anything with a blast radius runs.

The team has six weeks and two engineers. They already have a working retrieval assistant from the previous iteration.

What actually matters

An agent loop has the same shape wherever it’s implemented. The model is given a set of tools, each with a name, description, and input schema. The user asks something. The model returns either a tool call, with arguments, or text. The caller (us, or a framework, or Bedrock) invokes the tool, gets a result, feeds it back to the model. The next turn is another tool call, a question for the user, or a final answer. Repeat until done.

That framing exposes the decisions. The first is tool definition: how tools are described to the model, and how tightly their schemas are enforced. The second is invocation: when the model returns “call tool X with arguments Y,” who actually executes that? A Lambda? A local Python function? A remote service? The third is error recovery: when a tool fails, does the error go back as a tool result for the next turn, or does the whole conversation crash? The fourth is confirmation and guardrails: what stops a side-effecting action until the user has agreed, and what prevents refund running when the user said pause. The fifth is observability: traces of which tools ran, with what inputs, what outputs, how long, how much. The sixth is session state: conversation history and intermediate tool results need to persist across turns without ballooning the promptPromptThe input you hand to an LLM – system instructions, user message, examples, retrieved documents, tool descriptions, the lot..

The danger sits at the join. Model output is non-deterministic, and a tool that moves money cannot be invoked on non-deterministic output alone. Something outside the model has to hold the refund until a confirmation exists, and that something has to be a rule the loop cannot route around.

Then there’s debuggability in production. When an agent calls the wrong tool, passes the wrong arguments, or acts on the wrong customer ID, we need the steps it took, the tool inputs and outputs, the retry attempts, and the final user response. Not just at dev time; every production invocation.

What we’ll filter on

1. Tool-definition overhead, how much glue code per tool we write?
2. Side-effect safety, is confirmation enforced outside the agent or left to an instruction?
3. Observability out of the box, traces, metrics, replays without building it ourselves?
4. Flexibility, custom tool-selection logic, custom reasoning loops, dynamic tool sets?
5. Deployment shape, managed service, container, Lambda, all of the above?

The landscape

Bedrock AgentCore. A set of operational services for running agents, usable together or on their own. Runtime hosts the agent, each session in its own microVM with its own CPU, memory and filesystem, for up to eight hours. Gateway converts existing APIs, Lambda functions and MCP servers into Model Context Protocol tools, so the four internal endpoints become callable without being rewritten as bespoke actions, and it handles both inbound and outbound authentication. Identity manages workload identities and credential providers, so the agent reaches the subscriptions and billing APIs without a standing key in the code. Policy evaluates every call arriving at a Gateway against rules held outside the agent, before the tool runs. Observability emits OpenTelemetry spans and metrics to CloudWatch for each step. The reasoning loop can be ours, running on Runtime, or AgentCore’s own managed loop, Harness, which is configured rather than written. Framework-agnostic and model-agnostic; ticks all five.

A framework agent on our own infrastructure. LangChain or LangGraph, deployed to Lambda, Fargate, or EKS that we operate. LangChain defines tools as Python callables with type-annotated arguments; LangGraph models the agent as a directed graph of nodes. The loop is explicit code we own, but session isolation, credential brokering, and tracing come with it as work rather than as services, and every control lives in the same process as the loop. LangSmith traces are a separate subscription. Ticks 4 cleanly; 1, 3, and 5 become ours to build.

Custom tool router. We write the loop ourselves against a foundation model’s native tool-use API: the tools parameter on Anthropic’s Messages API, toolConfig on Bedrock’s Converse API, invoking tools with whatever runtime we like. The model returns a structured tool-use block; we parse it, run the tool, feed the result back as a tool-result block, repeat until the model returns plain text. Maximum control, maximum code, and every operational concern is ours. Ticks 4 entirely; gives up 1 and 3.

Step Functions + Bedrock. Not an agent, strictly. Step Functions as the orchestrator, Bedrock as a step, tool invocations as other steps. Works when the flow is largely deterministic with a language-model step in the middle: classify the request, then follow a hand-drawn state machine. It does not handle free-form multi-turn reasoning. Useful shape for certain problems; wrong shape for an open-ended support assistant.

Two shapes to rule out. Amazon Bedrock Agents, now Bedrock Agents Classic, is in maintenance mode and closed to new customers, so it is not available to a team starting today. The older chain pattern, a fixed sequence of model calls, is available and still wrong here: hard-coded control flow is not an agent, and the support assistant’s branching is wide enough that a chain would become a mess of if-statements.

Evaluation

Side by side

Reading it against the situation: side-effect safety is non-negotiable because a refund moves money, observability is non-negotiable because agent mistakes damage customer trust, and the flow is open-ended enough that Step Functions is the wrong shape. That leaves three. Two of them put the gate inside the process running the loop, where a bug in the agent code is a bug in the control. AgentCore evaluates the rule at the Gateway, outside the agent entirely, so rewriting the loop or talking the model into a different plan does not move it. With two engineers and six weeks, that plus the runtime, the credential management and the traces settles it.

The three agent loops, laid out

The solution

Tools through the Gateway. The four internal endpoints already exist behind OAuth2 client credentials, and they stay as they are. The Gateway converts them into MCP tools, so the agent discovers and calls subscriptions.lookup, subscriptions.pause, billing.refund, and notifications.sendEmail without any of them being rewritten as bespoke agent actions, and it handles the outbound token exchange for each. The schema each tool advertises is what the model reads before calling one, so the descriptions get the same care a public API reference would: what the tool does, what each argument means, and what it returns.

The confirmation gate, in Policy. A policy engine attached to the Gateway evaluates every call before the tool runs, deny by default, using rules written in Cedar or in Dogwood. Dogwood adds temporal conditions, which read what has already happened in the same session. So the rule for billing.refund permits it only when a matching approval for the same charge appears earlier in the session:

permit (
    principal,
    action == AgentCore::Action::"BillingTarget___Refund",
    resource == AgentCore::Gateway::"arn:aws:bedrock-agentcore:ap-southeast-2:123456789012:gateway/support"
)
when temporal {
    formerly within 1h AgentCore::Action::"ApprovalsTarget___RecordApproval"::response{
        eventResource:  resource,
        input.chargeId: context.input.chargeId,
        output.granted: true
    }
};

Our application still renders the confirmation button and calls the approval tool when the customer presses it. What it no longer does is decide whether the refund runs. That decision happens at the Gateway, on evidence the model cannot fabricate, and the same rule holds however the loop is rewritten.

The mechanics worth knowing before committing. The caller supplies a session ID on every request in the x-amzn-bedrock-agentcore-policy-session-id header; the Gateway does not generate one, and once a temporal rule is on the engine a request without it fails validation. History is scoped to that session, and a denied call is recorded as an error rather than a response, so the approval itself must be permitted for the refund rule to match it. An engine takes 25 temporal policies, each with at most three temporal operators over a window of at most 24 hours. Editing a temporal policy invalidates open sessions, and the next request on one returns a 409, so the application starts a fresh session. Run the rule in LOG_ONLY first and promote it to ENFORCE once the decisions look right. Temporal policies are not in every Region: Sydney and Singapore have them, N. California does not.

Identity. AgentCore Identity manages the workload identity and the credential providers for the calls the agent makes, instead of a standing key embedded in the code. The authenticated customer is the other half: their ID comes from the session the application established, never from an argument the model supplied. A prompt injection that gets userId: "u_999" into the arguments changes nothing: the agent code takes the customer ID from session context and drops the parameter, and a Cedar rule can forbid any call whose input identity differs from the principal, since Policy sees the tool’s input parameters as well as the caller.

Isolation, memory, and traces. Runtime executes each conversation in its own microVM, with its own CPU, memory and filesystem, for up to eight hours, and the microVM is destroyed at the end. AgentCore Memory holds short-term context within a session and long-term facts across sessions without us designing a datastore and a retention policy. Observability emits an OpenTelemetry span per step to CloudWatch: which tool was called with which arguments, what came back, how long it took, and where a run failed, alongside session, latency, token-usage and error metrics. When a customer says the assistant did the wrong thing, that trace is the answer.

Guardrails. Bedrock Guardrails screen both the input and the model’s response: denied topics, content filters across hate, insults, sexual, violence, misconduct and prompt attacks, word filters, and sensitive-information filters that either block or mask PII. ApplyGuardrail runs the same checks on arbitrary text without a model call, so the loop can screen a tool result before it goes back into context. Blocked content comes back as the guardrail’s configured message; masked content comes back with the entity replaced. One caveat: if model invocation logging is on, blocked content is still written to those logs in plain text.

Worked example

Customer: “I want to cancel my subscription and get a refund for the last month.”

1. The application starts a session, having already authenticated the customer as u_123, and generates a policy session ID it sends on every Gateway call. The agent gets the message, the Gateway’s tool list, and the system promptSystem promptThe instruction block that frames the model’s behaviour for a session, separate from the user’s messages..
2. The model asks for subscriptions.lookup. The rule for a read-only tool has no temporal condition, so Policy permits it, the Gateway calls the API with the session’s identity, and the subscription comes back.
3. The model asks for subscriptions.pause. No approval is on record for sub_xyz, so Policy denies the call and the agent gets an authorisation error rather than a paused subscription. The application renders “Confirm pausing subscription sub_xyz?” as a button.
4. The customer confirms. The application calls the approval tool, which records the grant as a session event; the model retries the pause; Policy finds the match and permits it.
5. The model asks for billing.refund on the last charge, AUD$49. Denied on the same grounds: “Confirm refunding AUD$49 of charge ch_abc?”
6. The customer confirms, the approval lands, the refund runs, and the ledger is written.
7. The model asks for notifications.sendEmail. Low blast radius, self-service, permitted with no condition attached.
8. The model produces a final response: the subscription is paused, AUD$49 refunded, confirmation email sent.

The trace holds every one of those calls, including the two denials, and the session can be replayed. The refund needed two recorded approvals, and neither of them came from anything the model emitted.

What’s worth remembering

1. An agent is a loop: call the model, call a tool, feed the result back, repeat. Every framework has one, so the work is defining tools well, gating side effects, and capturing traces.
2. A gate on a side-effecting tool belongs outside the loop, not in the prompt. AgentCore Policy evaluates each call at the Gateway, deny by default, so a rewritten loop or a talked-around model does not move the control.
3. A temporal policy is how “only after an approval” becomes a rule rather than an if-statement: it matches an approval event recorded earlier in the same policy session, and the caller supplies that session ID on every request.
4. Authenticated identity comes from the session, never from a tool argument. Read the customer ID from session context and drop whatever the model passed.
5. AgentCore is the operational half: microVM-isolated sessions, a Gateway that turns existing APIs into MCP tools, managed credentials, OTEL traces in CloudWatch, and a managed loop in Harness if you don’t want to write one.
6. Guardrails screen text, not actions. They stop a denied topic or leak PII; they will not stop a refund, and ApplyGuardrail is how you run them on a tool result mid-loop.

The assistant ships with four tools, three of them behind a policy, full traces, and a refund flow that needs the customer to press a button twice before money moves. The model can still ask for the wrong thing. The Gateway does not run it.