The situation
A wholesale distributor wants an assistant that answers questions from its own operations staff. “Where is order 40118?” “Did the substitution on the Tuesday run get approved?” “What did we ship this customer last month?” The model and the retrieval layer are the easy half. The hard half is that every fact worth answering with lives in systems the team is not permitted to modify.
The order system is on-premises and fifteen years old. It exposes SOAP endpoints over an internal network, was sized for a few dozen back-office users, and starts queueing above roughly five requests a second. It is offline from 01:00 to 04:00 every night for batch and index maintenance. The vendor that wrote it prices changes by the quarter and the internal team that knew it has moved on, so “add an endpoint” is not on the table this year or next.
Around that system sit three others. Customer records live in a SaaS CRM with a documented API. Signed delivery notes and proof-of-delivery scans land as PDFs on a Windows file share in the warehouse. The carrier pushes status updates as HTTP callbacks to a URL the distributor registered with them years ago, and will not accept being polled instead. The assistant has to draw on all four, and the ask that keeps coming back from operations is that it never be confidently wrong about what has actually shipped.
What actually matters
The first thing to weigh is what an assistant does to the traffic profile of whatever it touches. A back-office screen makes one call when a human clicks a button. A conversational assistant with tool access makes several per turn, often speculatively, because the model can decide to look something up, decide the result is not what it needed, and look again. Add a retrieval step that hydrates a few candidate records, then add ten operations staff asking questions at once. A system rated for a few dozen humans now sees an order of magnitude more calls, with no relationship to how many people are actually working. Any design that puts agent-rate traffic directly onto a fragile source has to answer for that before anything else.
The second is what staleness each answer can carry. This gets treated as one property of the system when it is really a property of each question. “What is our returns policy for chilled goods” tolerates a document that was accurate at midnight. “Has order 40118 left the depot” does not; an answer that is three hours old is worse than no answer, because the person asking will act on it. So the honest framing is not how fresh the integration is but which questions it is allowed to serve. A design that carries yesterday’s picture is fine as long as it never gets asked the live question. You enforce that by giving the assistant separate tools with separate contracts rather than one blurry pipe.
The third is coupling, in two directions. Availability coupling means that if the assistant calls the order system synchronously on every question, the assistant is down from 01:00 to 04:00 as well. It is down for the extra hour on the mornings when maintenance overruns, too. Change coupling means that a schema change on either side breaks the other. Both are why event-driven architectures to implement loose coupling keep coming up in enterprise connectivity solutions. A queue or an event bus in the middle means neither side has to be up at the same moment, and neither has to know the other’s shape.
The fourth is honesty about recency, which is a design constraint rather than a nicety. If any part of the answer came from a copy rather than the source, the assistant should be able to say when that copy was made. “As of 04:12 this morning, order 40118 was staged for the Tuesday run” is a useful sentence. The same fact with the timestamp stripped out reads as live and is the sentence that gets someone to send a truck. Whichever integration shape wins has to carry a watermark forward into the response, which means the pipeline has to record one.
What we’ll filter on
- Load on the source. How many calls per user question reach the system that cannot take them, and is there a ceiling that holds when the assistant misbehaves?
- Freshness. How old can the data be at answer time, and is that age known and reportable rather than assumed?
- Availability coupling. Does the assistant still answer during the maintenance window and during an unplanned source outage?
- Direction of initiative. Does the integration pull from the source, or does the source push? Some of these systems only do one.
- Coverage. Does the shape carry everything in the source, or only the subset the source chooses to emit or export?
- Recovery. After a missed window or a failed batch, how does the missing data get back in without a manual replay by hand?
The landscape
Four integration shapes cover this ground, and a real enterprise system integration usually ends up using more than one of them side by side.
A synchronous call straight through
Amazon API Gateway fronts a Lambda function that translates the assistant’s JSON tool call into a SOAP request, calls the on-premises endpoint over a private connection, and translates the XML response back. This is the plainest of the API-based integrations with legacy systems, and it uses API Gateway to implement microservice integrations in front of something that was never a microservice. Every answer is current to the millisecond.
The controls that make it survivable live in the layer in front. API Gateway usage plans and stage-level throttling give a hard ceiling on requests per second, so a runaway agent loop is rejected at the edge instead of arriving at the source. Response caching on the method collapses repeated lookups of the same order within a short window. Request validation rejects malformed tool calls against a model before Lambda is invoked at all, which costs nothing and keeps garbage off the wire. Reserved concurrency on the translating function is a second ceiling behind the first.
What it cannot do is answer during the maintenance window, and it inherits every latency spike the source has.
An event-driven integration
The source publishes changes and the generative-AI side reacts. Amazon EventBridge is the bus in the middle, an Amazon SQS queue buffers between the bus and the consumer, and a dead-letter queue catches what the consumer cannot process. Using Amazon EventBridge to implement event-driven integrations means the order system emits an order-dispatched event and stops caring who listens. The indexing Lambda, the notification path, and anything added later all attach to the same stream without the source knowing.
The buffer is what protects a slow consumer from a burst, and the dead-letter queue is what stops one poison message from stalling the queue behind it. EventBridge archive and replay is the recovery story. An archive on the bus keeps a window of events, and a replay pushes them back through the rules after an outage on the consumer side. A missed hour gets filled without anyone reconstructing it by hand.
The limit is coverage. This shape carries what the source chooses to emit and nothing else. A legacy system that publishes five event types gives you five event types, and a question about a sixth has no answer here.
A synchronised read copy
Data lands in Amazon S3 on a schedule and a Bedrock knowledge base indexes it, so the assistant reads a copy rather than the original. Which service does the landing depends on where the data lives, and these are the data synchronization patterns worth knowing by name. Amazon AppFlow moves records from SaaS sources such as the CRM on a schedule or on an event, with field mapping and filtering configured rather than coded. AWS DataSync moves files from on-premises NFS and SMB shares, which is what the warehouse file share of delivery-note PDFs needs. AWS Transfer Family stands up a managed SFTP, FTPS or AS2 endpoint in front of S3, which suits a partner or a legacy job that can only drop a file somewhere.
Once the data is in S3 the rest is familiar: a knowledge base over the bucket, and a sync schedule that decides how stale the index is allowed to get. Keeping that copy current is its own body of work, and the choice between copying data and calling for it live is the grounding trade-off in its usual form.
Load on the source is one scheduled read, which is why this shape is the one a fragile system can actually survive. The cost is that answers are as of the last sync.
An inbound webhook receiver
Some systems can push but cannot be polled, and the carrier here is one of them. API Gateway exposes an HTTPS endpoint and Lambda functions for webhook handlers do the work behind it: verify the signature, validate the body, write the delivery to durable storage, return 200 fast.
Three things separate a webhook handler that works from one that corrupts data quietly. Deliveries retry, so the handler must be idempotent on the provider’s delivery id: record the id, and treat a repeat as a no-op rather than a second status change. Deliveries arrive out of order, so the payload’s own event timestamp decides what is newest, not arrival order. And the handler should acknowledge before it does slow work, which usually means writing to SQS and returning, so a downstream stall does not turn into a delivery timeout and a retry storm. API Gateway request validation against a JSON schema rejects malformed payloads before Lambda spends anything, which matters when the sender is a third party you cannot get bug fixes from.
Evaluation
Side by side
| Shape | Load on source | Freshness | Survives maintenance window | Initiative | Coverage | Recovery after a gap |
|---|---|---|---|---|---|---|
| Synchronous call via API Gateway and translating Lambda | ✗ agent-rate | ✓ live | ✗ | Pull | ✓ whole API | ✓ nothing to recover |
| Event-driven via EventBridge, SQS and a DLQ | ✓ push only | ✓ near-live | ✓ | Push | ✗ emitted events only | ✓ archive and replay |
| Synced read copy via AppFlow, DataSync or Transfer Family into S3 | ✓ one scheduled read | ✗ as of last sync | ✓ | Pull, scheduled | ✓ whole export | ✓ next sync catches up |
| Inbound webhook receiver on API Gateway and Lambda | ✓ none | ✓ near-live | ✓ | Push | ✗ what the sender sends | ✗ needs a resend request |
No row is ticked everywhere, and the two columns that never agree are freshness and load. The only shape that answers the live question puts the traffic on the box that cannot take it, and the only shape a fragile source can survive answers from data that may be hours old.
Choosing by what the source can do
Most of the decision is made for you by the source rather than by preference, so the gates are worth walking in order.
The solution
The design that holds is all four shapes, each carrying the traffic it is suited to, with the assistant given four separate tools rather than one general-purpose bridge.
The bulk of the corpus arrives as a synchronised read copy. AppFlow pulls customer records from the CRM on an hourly flow into S3. DataSync runs a nightly task from the warehouse SMB share into a prefix of the same bucket, scheduled for 04:30 so it never overlaps the order system’s window. Transfer Family fronts an SFTP endpoint for the two trading partners who can only drop a file. A Bedrock knowledge base indexes the bucket and a sync runs after the last landing job of the night. Every object carries the sync timestamp as metadata, so a retrieved chunk can be attributed in the answer.
Change events from the order system go onto EventBridge. The five event types it can emit are enough to keep order state current between nightly copies: dispatched, delivered, cancelled, substituted, held. A rule routes them to an SQS queue, a consumer Lambda updates a DynamoDB projection of current order state, and a dead-letter queue takes what fails after the redrive policy gives up. An archive on the bus keeps fourteen days, so a consumer outage is repaired with a replay across the gap rather than a request to the vendor for a re-extract.
The synchronous path stays, narrowly. One tool, one operation, one order number, no list or search variants. It goes through API Gateway with a usage plan capping the assistant’s key at two requests per second and a burst of five, plus request validation on the tool payload. A short method cache stops a repeated lookup inside one conversation becoming a second SOAP call. The translating Lambda has reserved concurrency of five and a three-second timeout. During the maintenance window the tool returns a structured unavailable response rather than an error, and the assistant’s instructions tell it to fall back to the DynamoDB projection and say when that state was last updated.
Carrier callbacks land on the webhook endpoint. The handler verifies the shared-secret signature, checks the delivery id against a DynamoDB table with a conditional write, drops the payload on SQS, and returns 200. The idempotency check and the acknowledgement both happen before any downstream work, so a retried delivery is cheap and a slow consumer never causes one.
Two things tie it together. The assistant’s tool schemas are written so each one advertises its own freshness contract, an extension of what a safe tool schema already declares: the live lookup is described as current, the projection as last-updated-at, the knowledge base as as-of. And the response template requires the timestamp to be surfaced whenever an answer came from anything other than the live call. The queue-and-worker shape behind the events is the same one used for asynchronous document work, so the operational runbook for stuck queues already exists.
The gotcha that catches people is the maintenance window interacting with the nightly copy. Schedule the DataSync task inside 01:00 to 04:00 and it competes with batch on the same network path. Schedule the knowledge base sync before the copy has landed and you index yesterday’s files while believing you indexed today’s. Order the jobs and make each one depend on the last completing, rather than hoping the gaps are wide enough.
Worked example
Three questions arrive within a minute of each other.
“Where is order 40118?”
The model picks the live lookup tool. API Gateway validates the payload, the usage plan has headroom, the translating Lambda calls the SOAP endpoint and gets a response in 900ms. The answer is stated plainly with no timestamp, because it is current.
The same question at 02:40
The live tool returns its structured unavailable response. The assistant falls back to the DynamoDB projection, last written by an order-dispatched event at 00:51. It answers: staged for the Tuesday run as of 00:51. It adds that the order system is in its maintenance window, so the state may have moved since. The person asking now knows exactly how much to trust it.
“What did we ship this customer last month?”
No live tool covers this; the order system has no such operation and nobody is adding one. The knowledge base answers from the nightly export, and the answer carries “as of the 04:30 sync”. A month-old shipping history does not change overnight, so the staleness costs nothing here, which is why this question was routed to the copy in the first place.
What’s worth remembering
- Freshness and load on a fragile source pull against each other, and the shape you pick is mostly a decision about which questions are allowed to demand live data.
- A synchronous call through API Gateway to a translating Lambda is the only shape that answers live, so protect it with a usage plan, request validation, a method cache and reserved concurrency rather than trusting the agent to be gentle.
- EventBridge with an SQS buffer and a dead-letter queue decouples availability and change, and its archive-and-replay is how a missed window gets filled without a manual extract.
- AppFlow, DataSync and Transfer Family are the three landing paths into S3 for SaaS records, file shares and partner drops respectively, and a knowledge base over that bucket is what a fragile source can actually survive.
- Webhook handlers must be idempotent on the provider’s delivery id and acknowledge before doing slow work, because every provider retries and out-of-order delivery is normal.
- Give each tool its own freshness contract and make the assistant say as-of when the answer came from a copy, because an unqualified stale answer is the failure mode that gets someone to act on it.