The situation
An engineering consultancy of about three hundred staff runs a travel assistant inside its internal chat tool. Today it is one call to a foundation model on Amazon Bedrock: a system prompt carrying the travel policy, the staff member’s message, and whatever the model writes back. It handles roughly nine hundred requests a month.
Two thirds of those are questions. What is the per-diem in Singapore. Does a Sunday flight need approval. How far in advance do international trips have to be booked. The assistant answers them well, and the travel team has stopped fielding them.
The other third fail in the same way every time. Somebody types “book me Perth to Melbourne Tuesday morning, back Thursday evening, a hotel near the Collins Street office, and charge it to the Nakamura project”, and the assistant replies with a fluent paragraph describing the flights it would book. No flight is searched. No seat is held. No hotel is reserved, and the project code goes nowhere. The assistant writes about acting and never acts.
The travel team has asked for the assistant to do the booking. That sounds like a feature request. It is a change of category, and the change has consequences worth understanding before anyone builds it.
What actually matters
Begin with the difference between a model that answers and a model that acts. A single call is a function: text goes in, text comes out, and everything the model considers arrived in the prompt. Nothing outside the reply changes. An agent is that same model placed in a loop and given a set of tools it is allowed to call. It reads the request, chooses a tool, sees the result, and decides whether to call another tool or stop. That capacity to decide which tools to call, in what order, and when the work is finished, is what makes an application agentic AI. The label matters for governance as well as engineering, because a system that changes external state gets a different review from one that only produces text.
Autonomy costs four things, and being able to name all four is most of what this decision needs. The first is determinism. Ask a fixed program to book a flight twice and it performs the same steps twice. Ask an agent and it may search flights before checking policy on Monday and the other way round on Tuesday, both arriving at a defensible answer by different routes. That variation is the same property that lets it handle requests nobody anticipated.
The second is cost and latency. One request stops being one model call and becomes a sequence of them, one per turn of the loop, each carrying the conversation so far plus the tool results collected up to that point. A booking that takes eight turns is eight billed calls with a prompt that grows on each one, and the traveller waits for all of them. A single question that used to cost a fraction of a cent can become forty times that when it goes through an agent that did not need to.
The third is auditability. When a booking comes out wrong, somebody in the travel team has to reconstruct what the system did and why. With fixed steps that record is a log of steps that were always going to happen in that order. With an agent the record has to include which tools it chose, what it passed to them, what came back, and what it concluded, because none of that was decided in advance.
The fourth is reversibility, and it varies by tool rather than by system. Searching for the wrong flights costs nothing. Issuing a ticket on a non-refundable fare costs money and an apology. The tools an agent can reach divide cleanly into ones whose mistakes are free and ones whose mistakes are not, and that division does more work here than any question about which model to use.
Underneath all four sits a question about the work itself: how much does the order of the steps actually vary? A standard booking follows the same seven steps every time. A cancelled flight at 6am does not, because what to do next depends on what the airline offers, what the calendar says, and whether the hotel can be moved. One of those needs a system that can decide; the other only needs a system that can execute.
What we’ll filter on
- Order variability: is the sequence of steps the same on every request, or does the request decide it?
- Reversibility: when the system takes a wrong action, what does undoing it cost?
- Cost and latency: how many model calls does one request become, and how long does the person wait?
- Auditability: can someone reconstruct afterwards what was done, in what order, and on what basis?
- Memory horizon: what has to be remembered within one conversation, and what has to survive between them?
- Integration effort: what does connecting one more internal system cost each time?
The landscape
Five arrangements run from a single model call up to a system of cooperating agents. Each adds a capability to the one before it, and adds the costs that come with that capability. They are not five candidates competing for the same job either, because this assistant’s traffic ends up spread across three of them at once. The vocabulary first.
The words in play
| Term | What it means |
|---|---|
| tool usage | Giving the model a catalogue of functions it may call, with a description of each, so it can look things up and change things instead of only writing about them |
| memory management | Deciding what the system remembers within one conversation and what it carries between conversations, and for how long |
| workflow orchestration | Fixing the sequence of steps in advance, in code, and calling the model only for the parts that need language |
| multi-agent system patterns | Arrangements where several agents, each with its own tools and instructions, work on one request together |
| multi-agent communication patterns | How those agents pass work and results between them: a supervisor delegating to specialists, a handoff from one agent to the next, or a shared workspace they all read and write |
| Model Context Protocol [MCP] | An open protocol that lets an agent reach external tools and data through one common interface, instead of a bespoke integration per system |
One call to the model
Today’s arrangement. A prompt goes in, text comes out, and the model holds nothing between calls. Everything it appears to remember about the conversation is text the application put back into the prompt, which is why what fills the context window decides so much about how a chat assistant behaves. It is cheap, fast, predictable, and completely unable to change anything outside its own reply. For the two thirds of traffic that are questions, that is exactly the right shape.
Workflow orchestration
Ordinary application code holds the sequence, and for a booking the sequence is already known: check policy, search flights, present options, take a confirmation, issue the ticket, book the hotel, attach the project code. The model stays in the picture at the two places where language is the hard part, turning a free-text request into structured fields and writing the confirmation message back. It never chooses what comes next. Whoever wrote the code chose, months ago.
The system changes things in the world without giving up any of a single call’s predictability. Approvals sit at fixed points a policy owner can name, finance can be told what a booking costs before one is made, and the seven lines in the log arrive in the same order every time. The limit is exactly the thing that buys all of that: a request nobody anticipated has no branch waiting for it.
One agent with tool usage
Hand the model a catalogue of tools, each with a description and its expected inputs, then wrap the whole thing in a loop. Every turn ends in one of two ways: a tool call, or a final answer. The application executes whatever was asked for, returns the result, and runs the model again. No order is fixed anywhere, and those tool descriptions are all the model has to reason over when it picks. The mechanics of declaring tools and returning their results are the same whether the loop runs for two turns or twenty.
Strands Agents is the open-source SDK AWS provides for building one of these. A model, a set of tools, a prompt describing the job, and the loop comes with it. Amazon Bedrock AgentCore is the managed runtime for running agents in production, with per-session isolation, a memory store, and identity handling, independent of which framework or model you built with. Foundationally, the split is worth remembering as: the framework describes the agent, the runtime runs it. What a production runtime has to provide goes considerably deeper than this level needs.
Adding memory management
An agent that forgets everything the moment a conversation ends will ask a frequent traveller for the same preferences forty times a year. Memory management splits in two. Short-term memory is the state of the conversation in progress: what has been asked, which tools have already run, what they returned. Long-term memory is what survives between conversations: this traveller prefers an aisle seat, avoids red-eye flights, and always bills to one of three project codes.
The two carry different consequences. Short-term memory competes for space in the context window and gets trimmed or summarised as the conversation runs. Long-term memory is a data store with retention rules, a privacy question about what is kept, and a correctness question about stale facts, because a preference recorded in 2024 may no longer hold.
A multi-agent system
One request handled by several agents, each with its own instructions and its own narrower tool catalogue: a flights agent, a hotels agent, an expenses agent, and a supervisor that reads the request and delegates. These arrangements go by multi-agent system patterns, and the ways of handing work between the agents are multi-agent communication patterns. The common one is a supervisor delegating to specialists and assembling their replies. Others hand a conversation from one agent to the next, or give every agent a shared workspace to read from and write to.
The gain is that no single set of instructions has to describe every tool, so each agent stays reliable at its own job. The cost is a request that now involves several loops instead of one, more model calls, more latency, and a failure mode where two agents disagree and nobody has been told who wins. Coordinating several agents in production is a whole engineering topic on its own.
Model Context Protocol, underneath the last three
MCP sits beneath the agent options rather than beside them. Without it, every internal system an agent reaches gets its own bespoke wiring: a hand-written tool definition, a hand-written adapter, and a maintenance burden that grows with each system added. Model Context Protocol is an open protocol that standardises that connection. A system exposes its capabilities once as an MCP server, and any agent that speaks the protocol can discover and call them through the same interface.
For the travel assistant that turns “connect the expenses system” from a bespoke integration into pointing the agent at another server. It changes the integration cost, not the reasoning: the agent still has to choose well among the tools it now has.
Evaluation
Side by side
| Option | Copes with variable order | Reversible by design | Low cost and latency | Straightforward to audit | Memory across turns |
|---|---|---|---|---|---|
| One call to the model | ✗ | ✓ | ✓ | ✓ | ✗ |
| Workflow orchestration | ✗ | ✓ | ✓ | ✓ | ✗ |
| One agent with tool usage | ✓ | ✗ | ✗ | ✗ | ✗ |
| Agent with memory management | ✓ | ✗ | ✗ | ✗ | ✓ |
| Multi-agent system | ✓ | ✗ | ✗ | ✗ | ✓ |
Read the first column against the rest. Everything that copes with a request whose steps were not known in advance gives up predictability, cheapness and an easy audit trail to do it. Nothing on this list gives both, and no configuration setting will change that, which is why the choice comes down to whether the work in front of you genuinely varies.
The reversibility column is the one that can be moved. It reads ✗ for every agent row because an agent chooses its own actions, but placing a human confirmation in front of the small number of tools that spend money or issue tickets restores most of it, at the cost of the agent no longer running end to end.
Integration effort has no column because MCP moves it for every agent row at once and moves it in the same direction. It affects what building costs rather than which arrangement suits the work.
Where the choice branches
The chart matters because the assistant’s traffic does not take one path through it. Two thirds of requests stop at the first gate. Most of the rest reach the second and stop there. A small tail goes all the way down. Treating nine hundred requests a month as one workload and building the most capable thing for all of them is how a chat assistant ends up costing forty times what it needs to.
The solution
Split the traffic and give each part the simplest arrangement that handles it.
The questions stay as they are. One call to the model, the travel policy in the system prompt, no tools and no loop. Nothing about adding an agent elsewhere is a reason to route policy questions through one.
Standard bookings become workflow orchestration. The seven steps are the same every time, so they get written down: parse the request into structured fields, check it against policy, search flights, present the options, take the traveller’s confirmation, issue the ticket and book the hotel, attach the project code and file the pre-approval. The model is called at the first step, where free text becomes fields, and at the last, where a confirmation message gets written. Everything between is ordinary code. Cost per booking is known in advance, the log is the same seven lines every time, and the human confirmation sits at the one place where the next action spends money.
The tail, the disruptions and rebookings and awkward multi-city requests, gets one agent with tool usage. Define it with Strands Agents: a model, a set of tools covering flight search, seat holds, hotel availability, calendar lookup and expense filing, and instructions describing the job and its limits. Run it on Amazon Bedrock AgentCore so each traveller’s session is isolated and the memory store is somebody else’s operational problem. Memory management is configured in two halves: the conversation in progress as short-term state, and a small long-term record per traveller holding seat preference, frequent-flyer numbers and usual project codes. Choosing the framework you define an agent in is a separate decision from choosing where it runs.
Four things go wrong often enough to build against from the start. Bound the loop with a maximum number of turns and a token budget, because an agent that misreads a tool error can call it repeatedly until the budget is gone. Separate read tools from write tools and give them different permissions, so a mistaken plan can search everything and book nothing. Log every tool call with its inputs and its outputs, not only the final reply, because the reply is the one part of the record that was written by a model. And check the write tools’ actual return values rather than trusting the agent’s summary, because an agent that believes it booked a flight will say so with complete confidence.
Leave the multi-agent system alone for now. At nine hundred requests a month one agent’s tool catalogue is small enough to describe reliably in one set of instructions. The signal to revisit is a catalogue that has grown until the agent starts picking the wrong tool, or a part of the request that needs different permissions or a different model from the rest. At that point a supervisor delegating to a flights specialist and an expenses specialist starts to earn the extra calls, and the vocabulary of agents and orchestration is worth having ready before that conversation.
Worked example
The standard booking
“Book me Perth to Melbourne Tuesday morning, back Thursday evening, hotel near Collins Street, Nakamura project.” One model call turns that into fields: origin, destination, two dates, a time preference on each, a hotel area, a project code. Application code checks the fare class against policy and calls the flight search API. Three options come back and are shown to the traveller, who picks one. Their click is the confirmation, so the code issues the ticket, books the hotel from the same shortlist logic, files the pre-approval against the project code, and calls the model once more to write the confirmation. Two model calls, a fixed sequence, and an audit log that reads the same as every other booking.
The cancelled flight
“My Thursday flight has been cancelled, sort it out.” Nothing here can be sequenced in advance, because what to do second depends on what the first tool returns. The agent looks up the booking, finds the cancellation, and checks what the airline is offering. There is a seat on the 6am, but long-term memory says this traveller avoids departures before 8am, so it checks the afternoon service, finds space, and looks at the calendar to see whether the Thursday meeting can be moved. It can. It then checks whether the hotel can extend by one night, finds it can, and stops: rebooking the flight and extending the hotel both spend money, so it presents both changes and waits. The traveller confirms, and the two write tools run. Nine turns, seven tool calls, one human decision at the one place it mattered.
What’s worth remembering
- An application is agentic AI when the model decides which tools to call, in what order, and when the work is finished, rather than returning one answer to one prompt.
- Autonomy costs determinism, cost per request, latency and auditability, so take it only where the order of the steps genuinely varies from request to request.
- Workflow orchestration fixes the sequence in code and calls the model only for the language-shaped steps, which suits any process whose steps are the same every time.
- Memory management splits into short-term conversation state, which competes for the context window, and longer-term recall between conversations, which carries retention and privacy consequences.
- Model Context Protocol (MCP) gives an agent one common interface to external tools and data, so connecting another system stops meaning another bespoke integration.
- Multi-agent system patterns spread one request across specialists coordinated by a supervisor, and the extra model calls are worth paying once a single tool catalogue has grown too large for one agent to choose from reliably.