The situation
An engineering consultancy of about three hundred staff runs a travel assistant inside its internal chat tool. Today it is one call to a foundation model on Amazon Bedrock: a system prompt carrying the travel policy, the staff member’s message, and whatever the model writes back. It handles roughly nine hundred requests a month.
Two thirds of those are questions. What is the per-diem in Singapore. Does a Sunday flight need approval. How far in advance do international trips have to be booked. The assistant answers them well, and the travel team has stopped fielding them.
The other third fail in the same way every time. Somebody types “book me Perth to Melbourne Tuesday morning, back Thursday evening, a hotel near the Collins Street office, and charge it to the Nakamura project”, and the assistant replies with a fluent paragraph describing the flights it would book. No flight is searched. No seat is held. No hotel is reserved, and the project code goes nowhere. The assistant writes about acting and never acts.
The travel team has asked for the assistant to do the booking. That sounds like a feature request. It is a change of category, and the change has consequences worth understanding before anyone builds it.
What actually matters
Begin with the difference between a model that answers and a model that acts. A single call is a function: text goes in, text comes out, and everything the model works from arrived in the prompt. Nothing outside the reply changes. An agent is that same model placed in a loop and given a set of tools it is allowed to call. Each turn produces either a tool call or a final answer. The application runs the tool, feeds the result back, and runs the model again. That arrangement, where the model’s output selects which tools run, in what order, and when the work is finished, is what makes an application agentic AI. The label matters for governance as well as engineering, because a system that changes external state gets a different review from one that only produces text.
Autonomy changes four things, and being able to name all four is most of what this decision needs. The first is determinism. Ask a fixed program to book a flight twice and it performs the same steps twice. Ask an agent and it may search flights before checking policy on Monday and the other way round on Tuesday, both arriving at a defensible answer by different routes. That variation is the same property that lets it handle requests nobody anticipated.
The second is cost and latency. One request stops being one model call and becomes a sequence of them, one per turn of the loop, each carrying the conversation so far plus the tool results collected up to that point. A booking that takes eight turns is eight billed calls with a prompt that grows on each one, and the traveller waits for all of them. Because every turn re-sends everything before it, the tokens billed for one request climb faster than the turn count does. Sending a question through that loop when one call would have answered it multiplies the bill and the wait for nothing.
The third is auditability. When a booking comes out wrong, somebody in the travel team has to reconstruct what the system did and why. With fixed steps that record is a log of steps that were always going to happen in that order. With an agent the record has to include which tools ran, what was passed to them, what came back, and what the model output at each turn, because none of that was fixed in advance.
The fourth is reversibility, and it varies by tool rather than by system. Searching for the wrong flights changes nothing. Issuing a ticket on a non-refundable fare moves money and needs an apology. The tools an agent can reach divide cleanly into ones whose mistakes can be undone and ones whose mistakes cannot, and that division does more work here than any question about which model to use.
Underneath all four sits a question about the work itself: how much does the order of the steps actually vary? A standard booking follows the same seven steps every time. A cancelled flight at 6am does not, because what to do next depends on what the airline offers, what the calendar says, and whether the hotel can be moved. One of those needs a system that works out the order as it goes; the other only needs one that follows a known order.
What we’ll filter on
- Order variability: is the sequence of steps the same on every request, or does the request decide it?
- Reversibility: when the system takes a wrong action, what does undoing it cost?
- Cost and latency: how many model calls does one request become, and how long does the person wait?
- Auditability: can someone reconstruct afterwards what was done, in what order, and on what basis?
- Memory horizon: what has to be remembered within one conversation, and what has to survive between them?
- Integration effort: how much work is connecting one more internal system, each time?
The landscape
Five arrangements run from a single model call up to a system of cooperating agents. Each adds a capability to the one before it, and brings the trade-offs that come with it. They are not five candidates competing for the same job either, because this assistant’s traffic ends up spread across three of them at once.
The words in play
| Term | What it means |
|---|---|
| tool usage | Giving the model a catalogue of functions it may call, with a description of each, so the system can look things up and change things instead of only writing about them |
| memory management | Deciding what the system remembers within one conversation and what it carries between conversations, and for how long |
| workflow orchestration | Fixing the sequence of steps in advance, in code, and calling the model only for the parts that need language |
| multi-agent system patterns | Arrangements where several agents, each with its own tools and instructions, work on one request together |
| multi-agent communication patterns | How those agents pass work and results between them: a supervisor delegating to specialists, a handoff from one agent to the next, or a shared workspace they all read and write |
| Model Context Protocol [MCP] | An open protocol that lets an agent reach external tools and data through one common interface, instead of a bespoke integration per system |
One call to the model
Today’s arrangement. A prompt goes in, text comes out, and nothing is held between calls. Everything that looks like memory of the conversation is text the application put back into the prompt, which is why what fills the context window determines so much about how a chat assistant behaves. It is cheap, fast, predictable, and unable to change anything outside its own reply. For the two thirds of traffic that are questions, that is exactly the right shape.
Workflow orchestration
Ordinary application code holds the sequence, and for a booking the sequence is already known: check policy, search flights, present options, take a confirmation, issue the ticket, book the hotel, attach the project code. The model stays in the picture at the two places where language is the hard part, turning a free-text request into structured fields and writing the confirmation message back. Nothing about what comes next is settled at run time. Whoever wrote the code settled it, months ago.
The system changes things in the world without giving up any of a single call’s predictability. Approvals sit at fixed points a policy owner can name, finance can be told what a booking costs before one is made, and the seven lines in the log arrive in the same order every time. The limit comes from the same fixed sequence that delivers all of that: a request nobody anticipated has no branch waiting for it.
One agent with tool usage
Hand the model a catalogue of tools, each with a description and its expected inputs, then wrap the whole thing in a loop. Every turn ends in one of two ways: a tool call, or a final answer. On Bedrock the application usually executes the call and returns the result in the next request; Bedrock can also run the tool itself, from a registered Lambda function or an AgentCore Gateway. Either way no order is fixed anywhere, and those tool descriptions are the only information the model has when it selects one. The mechanics of declaring tools and returning their results are the same whether the loop runs for two turns or twenty.
Strands Agents is an open-source SDK, first released by AWS, for building one of these. A model, a set of tools, a prompt describing the job, and the loop comes with it. Amazon Bedrock AgentCore is the managed platform for running agents in production, a set of services used together or separately: a serverless Runtime with per-session isolation and built-in identity, a Memory store, and a Gateway that turns existing APIs and Lambda functions into MCP tools. It works with any framework, Strands and LangGraph and CrewAI among them, and any foundation model, inside Bedrock or outside it. The split worth remembering is that the framework describes the agent and the platform runs it. What a production runtime has to provide goes deeper than this level needs.
Adding memory management
An agent that keeps nothing past the end of a conversation will ask a frequent traveller for the same preferences forty times a year. Memory management splits in two. Short-term memory is the state of the conversation in progress: what has been asked, which tools have already run, what they returned. Long-term memory is what survives between conversations: this traveller prefers an aisle seat, avoids red-eye flights, and always bills to one of three project codes.
The two carry different consequences. Short-term memory occupies space in the context window and gets trimmed or summarised as the conversation runs. Long-term memory is a data store with retention rules, a privacy question about what is kept, and a correctness question about stale facts, because a preference recorded in 2024 may no longer hold.
A multi-agent system
One request handled by several agents, each with its own instructions and its own narrower tool catalogue: a flights agent, a hotels agent, an expenses agent, and a supervisor that reads the request and delegates. These arrangements go by multi-agent system patterns, and the ways of handing work between the agents are multi-agent communication patterns. The common one is a supervisor delegating to specialists and assembling their replies. Others hand a conversation from one agent to the next, or give every agent a shared workspace to read from and write to.
The gain is that no single set of instructions has to describe every tool, so each agent stays reliable at its own job. The trade is a request that now involves several loops instead of one, more model calls, more latency, and a failure mode where two agents return conflicting answers and nothing settles which one applies. Coordinating several agents in production is a whole engineering topic on its own.
Model Context Protocol, underneath the last three
MCP sits beneath the agent options rather than beside them. Without it, every internal system an agent reaches gets its own bespoke wiring: a hand-written tool definition, a hand-written adapter, and a maintenance burden that grows with each system added. Model Context Protocol is an open protocol that standardises that connection. A system exposes its capabilities once as an MCP server, and any agent that speaks the protocol can discover and call them through the same interface.
For the travel assistant that turns “connect the expenses system” from a bespoke integration into pointing the agent at another server. The wiring work shrinks. The selection problem does not: the model still has to pick well among the tools it now has.
Evaluation
Side by side
| Option | Copes with variable order | Reversible by design | Low cost and latency | Straightforward to audit | Memory across turns |
|---|---|---|---|---|---|
| One call to the model | ✗ | ✓ | ✓ | ✓ | ✗ |
| Workflow orchestration | ✗ | ✓ | ✓ | ✓ | ✗ |
| One agent with tool usage | ✓ | ✗ | ✗ | ✗ | ✗ |
| Agent with memory management | ✓ | ✗ | ✗ | ✗ | ✓ |
| Multi-agent system | ✓ | ✗ | ✗ | ✗ | ✓ |
Read the first column against the rest. Everything that copes with a request whose steps were not known in advance gives up predictability, low cost and an easy audit trail to do it. No row ticks both sides, and no configuration setting will produce one, so the choice comes down to whether the work in front of you actually varies.
The reversibility column is the one that can be moved. It reads ✗ for every agent row because the model selects the actions at run time, but placing a human confirmation in front of the small number of tools that spend money or issue tickets restores most of it. What goes is the agent running end to end without a person in the loop.
Integration effort has no column because MCP moves it for every agent row at once and moves it in the same direction. It changes how much work building takes, not which arrangement suits the work.
Where the choice branches
The assistant’s traffic does not take one path through this chart. Two thirds of requests stop at the first gate. Most of the rest reach the second and stop there. A small tail goes all the way down. Treating nine hundred requests a month as one workload and building the most capable thing for all of them runs every policy question through a loop that bills several calls to answer what one call answered.
The solution
Split the traffic and give each part the simplest arrangement that handles it.
The questions stay as they are. One call to the model, the travel policy in the system prompt, no tools and no loop. Nothing about adding an agent elsewhere is a reason to route policy questions through one.
Standard bookings become workflow orchestration. The seven steps are the same every time, so they get written down: parse the request into structured fields, check it against policy, search flights, present the options, take the traveller’s confirmation, issue the ticket and book the hotel, attach the project code and file the pre-approval. The model is called at the first step, where free text becomes fields, and at the last, where a confirmation message gets written. Everything between is ordinary code. Cost per booking is known in advance, the log is the same seven lines every time, and the human confirmation sits at the one place where the next action spends money.
The tail, the disruptions and rebookings and awkward multi-city requests, gets one agent with tool usage. Define it with Strands Agents: a model, a set of tools covering flight search, seat holds, hotel availability, calendar lookup and expense filing, and instructions describing the job and its limits. Run it on the AgentCore Runtime so each traveller’s session is isolated, and use AgentCore Memory rather than building a store. Memory management is configured in two halves: the conversation in progress as short-term state, and a small long-term record per traveller holding seat preference, frequent-flyer numbers and usual project codes. Choosing the framework you define an agent in is a separate decision from choosing where it runs.
Four things go wrong often enough to build against from the start. Bound the loop with a maximum number of turns and a token budget, because a tool error the model does not handle can produce the same call again and again until the budget is gone. Separate read tools from write tools and give them different permissions, so a mistaken plan can search everything and book nothing. Log every tool call with its inputs and its outputs, not only the final reply, because the reply is the one part of the record that was written by a model. And check the write tools’ actual return values rather than the agent’s summary, because the model can report a booking in fluent prose when the write tool never completed one.
Leave the multi-agent system alone for now. At nine hundred requests a month one agent’s tool catalogue is small enough to describe reliably in one set of instructions. The signal to revisit is a catalogue that has grown until the agent starts picking the wrong tool, or a part of the request that needs different permissions or a different model from the rest. At that point a supervisor delegating to a flights specialist and an expenses specialist justifies the extra calls, and the vocabulary of agents and orchestration is worth having ready before that conversation.
Worked example
The standard booking
“Book me Perth to Melbourne Tuesday morning, back Thursday evening, hotel near Collins Street, Nakamura project.” One model call turns that into fields: origin, destination, two dates, a time preference on each, a hotel area, a project code. Application code checks the fare class against policy and calls the flight search API. Three options come back and are shown to the traveller, who picks one. Their click is the confirmation, so the code issues the ticket, books the hotel from the same shortlist logic, files the pre-approval against the project code, and calls the model once more to write the confirmation. Two model calls, a fixed sequence, and an audit log that reads the same as every other booking.
The cancelled flight
“My Thursday flight has been cancelled, sort it out.” Nothing here can be sequenced in advance, because what to do second depends on what the first tool returns. The agent looks up the booking, finds the cancellation, and checks what the airline is offering. There is a seat on the 6am, but long-term memory says this traveller avoids departures before 8am, so it checks the afternoon service, finds space, and looks at the calendar to see whether the Thursday meeting can be moved. It can. It then checks whether the hotel can extend by one night, finds it can, and stops: rebooking the flight and extending the hotel both spend money, so it presents both changes and waits. The traveller confirms, and the two write tools run. Nine turns, seven tool calls, one human decision at the one place it mattered.
What’s worth remembering
- An application is agentic AI when the model’s output selects which tools run, in what order, and when the work is finished, rather than returning one answer to one prompt.
- Autonomy gives up determinism and a straightforward audit trail, and adds calls and latency, so take it only where the order of the steps varies from request to request.
- Workflow orchestration fixes the sequence in code and calls the model only for the language-shaped steps, which suits any process whose steps are the same every time.
- Memory management splits into short-term conversation state, which competes for the context window, and longer-term recall between conversations, which carries retention and privacy consequences.
- Model Context Protocol (MCP) gives an agent one common interface to external tools and data, so connecting another system stops meaning another bespoke integration.
- Multi-agent system patterns spread one request across specialists coordinated by a supervisor, which justifies the extra model calls once a single tool catalogue has grown too large for one agent to select from reliably.