The situation
A wholesale distributor wants an assistant that answers questions from its own operations staff. “Where is order 40118?” “Did the substitution on the Tuesday run get approved?” “What did we ship this customer last month?” The model and the retrieval layer are the easy half. The hard half is that every fact worth answering with lives in systems the team is not permitted to modify.
The order system is on-premises and fifteen years old. It exposes SOAP endpoints over an internal network, was sized for a few dozen back-office users, and starts queueing above roughly five requests a second. It is offline from 01:00 to 04:00 every night for batch and index maintenance. The vendor that wrote it prices changes by the quarter and the internal team that knew it has moved on, so “add an endpoint” is not on the table this year or next.
Around that system sit three others. Customer records live in a SaaS CRM with a documented API. Signed delivery notes and proof-of-delivery scans land as PDFs on a Windows file share in the warehouse. The carrier pushes status updates as HTTP callbacks to a URL the distributor registered with them years ago, and will not accept being polled instead. The assistant has to draw on all four, and the ask that keeps coming back from operations is that it never be confidently wrong about what has actually shipped.
What actually matters
The first thing to weigh is what an assistant does to the traffic profile of whatever it touches. A back-office screen makes one call when a human clicks a button. A conversational assistant with tool access makes several per turn, often speculatively, because a tool-calling loop issues a lookup, gets a result that does not satisfy the request, and issues another. Add a retrieval step that hydrates a few candidate records, then add ten operations staff asking questions at once. A system rated for a few dozen humans now sees an order of magnitude more calls, with no relationship to how many people are actually working. Any design that puts agent-rate traffic directly onto a fragile source has to answer for that before anything else.
The second is what staleness each answer can carry. This gets treated as one property of the system when it is really a property of each question. “What is our returns policy for chilled goods” tolerates a document that was accurate at midnight. “Has order 40118 left the depot” does not; an answer that is three hours old is worse than no answer, because the person asking will act on it. So the thing to settle is which questions each integration is allowed to serve. A design that carries yesterday’s picture is fine as long as it never gets asked the live question. You enforce that by giving the assistant separate tools with separate contracts rather than one blurry pipe.
The third is coupling, in two directions. Availability coupling means that if the assistant calls the order system synchronously on every question, the assistant is down from 01:00 to 04:00 as well. It is down for the extra hour on the mornings when maintenance overruns, too. Change coupling means that a schema change on either side breaks the other. Both are why a queue or an event bus in the middle keeps coming up in enterprise connectivity work. Neither side has to be up at the same moment, and a schema change on one side stops at the bus instead of reaching the other.
The fourth is honesty about recency, which is a design constraint rather than a nicety. If any part of the answer came from a copy rather than the source, the assistant should be able to say when that copy was made. “As of 04:12 this morning, order 40118 was staged for the Tuesday run” is a useful sentence. The same fact with the timestamp stripped out reads as live and is the sentence that gets someone to send a truck. Whichever integration shape wins has to carry a watermark forward into the response, which means the pipeline has to record one.
What we’ll filter on
- Load on the source. How many calls per user question reach the system that cannot take them, and is there a ceiling that holds when the assistant misbehaves?
- Freshness. How old can the data be at answer time, and is that age known and reportable rather than assumed?
- Availability coupling. Does the assistant still answer during the maintenance window and during an unplanned source outage?
- Direction of initiative. Does the integration pull from the source, or does the source push? Some of these systems only do one.
- Coverage. Does the shape carry everything in the source, or only the subset the source is built to emit or export?
- Recovery. After a missed window or a failed batch, how does the missing data get back in without someone reconstructing it?
The landscape
Four integration shapes cover this ground, and a real enterprise system integration usually ends up using more than one of them side by side.
A synchronous call straight through
Amazon API Gateway fronts a Lambda function that translates the assistant’s JSON tool call into a SOAP request, calls the on-premises endpoint over a private connection, and translates the XML response back. This is the plainest of the API-based integrations with a legacy system: API Gateway gives a modern request shape to something that was never built to offer one. Every answer reflects the source at the moment the call lands.
The controls that make it survivable live in the layer in front, and all three are REST API features. A usage plan caps rate and burst per API key, and stage or method throttling caps them for everyone, so a runaway tool-calling loop meets a 429 at the edge rather than arriving at the source. Throttling is applied on a best-effort basis, so treat the number as a target and put a second limit behind it: reserved concurrency on the translating function, which is both the floor and the ceiling on how many copies run at once. Stage caching collapses repeated lookups of the same order, at a default TTL of 300 seconds and a maximum of 3,600, though only GET methods are cached until you override the method setting. Request validation against a JSON schema model returns 400 before the integration request, so a malformed tool call never reaches Lambda or the SOAP endpoint.
What it cannot do is answer during the maintenance window, and it inherits every latency spike the source has.
An event-driven integration
The source publishes changes and the generative-AI side reacts. Amazon EventBridge is the bus in the middle, an Amazon SQS queue buffers between the bus and the consumer, and a dead-letter queue catches what the consumer cannot process. The order system emits an order-dispatched event and keeps no list of consumers. The indexing Lambda, the notification path, and anything added later all attach to the same bus, and none of it requires a change at the source.
The buffer is what protects a slow consumer from a burst, and the dead-letter queue is what stops one poison message from stalling the queue behind it. EventBridge archive and replay is the recovery story. An archive on the bus retains events for a number of days you set, and a replay sends them back to that same bus, through all its rules or through named ones, after an outage on the consumer side. A missed hour gets filled without anyone reconstructing it. Replayed events carry a replay-name field and do not arrive in their original order, because a replay works through the chosen window a minute at a time, so the consumer has to be idempotent and order-tolerant regardless.
The limit is coverage. This shape carries whatever the source is built to emit and nothing else. A legacy system that publishes five event types gives you five event types, and a question about a sixth has no answer here.
A synchronised read copy
Data lands in Amazon S3 on a schedule and a Bedrock knowledge base indexes it, so the assistant reads a copy rather than the original. Which service does the landing depends on where the data lives. Amazon AppFlow moves records from SaaS sources such as the CRM on a schedule or on an event, with field mapping and filtering configured rather than coded. AWS DataSync moves files from on-premises NFS and SMB shares, which is what the warehouse file share of delivery-note PDFs needs. AWS Transfer Family stands up a managed SFTP, FTPS, FTP or AS2 endpoint in front of S3 or EFS, which suits a partner or a legacy job that can only drop a file somewhere.
Once the data is in S3 the rest is familiar: a knowledge base over the bucket, and a sync schedule that sets how stale the index is allowed to get. Keeping that copy current is its own body of work, and the choice between copying data and calling for it live is the grounding trade-off in its usual form.
Load on the source is one scheduled read, which is what makes this shape survivable for a fragile system. Answers are only ever as of the last sync.
An inbound webhook receiver
Some systems can push but cannot be polled, and the carrier here is one of them. A REST API on API Gateway exposes an HTTPS endpoint and Lambda functions for webhook handlers do the work behind it: verify the signature, validate the body, write the delivery to durable storage, return 200 fast.
Three things separate a webhook handler that works from one that corrupts data without raising an error. Deliveries retry, so the handler must be idempotent on the provider’s delivery id: record the id, and treat a repeat as a no-op rather than a second status change. Deliveries arrive out of order, so use the payload’s own event timestamp rather than arrival order to tell what is newest. And the handler should acknowledge before it does slow work, which usually means writing to SQS and returning, so a downstream stall does not turn into a delivery timeout and a retry storm. Request validation against a JSON schema returns 400 on a malformed payload before the integration request, which matters when the sender is a third party you cannot get bug fixes from.
Evaluation
Side by side
| Shape | Load on source | Freshness | Survives maintenance window | Initiative | Coverage | Recovery after a gap |
|---|---|---|---|---|---|---|
| Synchronous call via API Gateway and translating Lambda | ✗ agent-rate | ✓ live | ✗ | Pull | ✓ whole API | ✓ nothing to recover |
| Event-driven via EventBridge, SQS and a DLQ | ✓ push only | ✓ near-live | ✓ | Push | ✗ emitted events only | ✓ archive and replay |
| Synced read copy via AppFlow, DataSync or Transfer Family into S3 | ✓ one scheduled read | ✗ as of last sync | ✓ | Pull, scheduled | ✓ whole export | ✓ next sync catches up |
| Inbound webhook receiver on API Gateway and Lambda | ✓ none | ✓ near-live | ✓ | Push | ✗ what the sender sends | ✗ needs a resend request |
No row is ticked everywhere, and the two columns that never agree are freshness and load. The only shape that answers the live question puts the traffic on the box that cannot take it, and the only shape a fragile source can survive answers from data that may be hours old.
Choosing by what the source can do
Most of the decision is made for you by the source rather than by preference, so the gates are worth walking in order.
The solution
The design that holds is all four shapes, each carrying the traffic it is suited to, with the assistant given four separate tools rather than one general-purpose bridge.
The bulk of the corpus arrives as a synchronised read copy. AppFlow pulls customer records from the CRM on an hourly flow into S3. DataSync runs a nightly task from the warehouse SMB share into a prefix of the same bucket, scheduled for 04:30 so it never overlaps the order system’s window. Transfer Family fronts an SFTP endpoint for the two trading partners who can only drop a file. A Bedrock knowledge base indexes the bucket and a sync runs after the last landing job of the night. Each landed file gets a sidecar name.extension.metadata.json in the same prefix carrying the sync timestamp, within the 10 KB limit those files have, so a retrieved chunk can be attributed in the answer.
Change events from the order system go onto EventBridge. The five event types it can emit are enough to keep order state current between nightly copies: dispatched, delivered, cancelled, substituted, held. A rule routes them to an SQS queue, a consumer Lambda updates a DynamoDB projection of current order state, and a dead-letter queue takes what fails once the redrive policy’s maxReceiveCount is exceeded. An archive on the bus is set to retain fourteen days, so a consumer outage is repaired with a replay across the gap rather than a request to the vendor for a re-extract.
The synchronous path stays, narrowly. One tool, one operation, one order number, no list or search variants. It goes through API Gateway with a usage plan capping the assistant’s key at two requests per second and a burst of five, plus request validation on the tool payload. A short method cache stops a repeated lookup inside one conversation becoming a second SOAP call. The translating Lambda has reserved concurrency of five and a three-second timeout. During the maintenance window the tool returns a structured unavailable response rather than an error, and the assistant’s instructions tell it to fall back to the DynamoDB projection and say when that state was last updated.
Carrier callbacks land on the webhook endpoint. The handler verifies the shared-secret signature, checks the delivery id against a DynamoDB table with a conditional write, drops the payload on SQS, and returns 200. The idempotency check and the acknowledgement both happen before any downstream work, so a retried delivery does no work twice and a slow consumer never causes a retry.
Two things tie it together. The assistant’s tool schemas are written so each one advertises its own freshness contract, an extension of what a safe tool schema already declares: the live lookup is described as current, the projection as last-updated-at, the knowledge base as as-of. And the response template requires the timestamp to be surfaced whenever an answer came from anything other than the live call. The queue-and-worker shape behind the events is the same one used for asynchronous document work, so the operational runbook for stuck queues already exists.
The gotcha that catches people is the maintenance window interacting with the nightly copy. Schedule the DataSync task inside 01:00 to 04:00 and it competes with batch on the same network path. Schedule the knowledge base sync before the copy has landed and you index yesterday’s files while believing you indexed today’s. Order the jobs and make each one depend on the last completing, rather than hoping the gaps are wide enough.
Worked example
Three questions arrive within a minute of each other.
“Where is order 40118?”
The model picks the live lookup tool. API Gateway validates the payload, the usage plan has headroom, the translating Lambda calls the SOAP endpoint and gets a response in 900ms. The answer is stated plainly with no timestamp, because it is current.
The same question at 02:40
The live tool returns its structured unavailable response. The assistant falls back to the DynamoDB projection, last written by an order-dispatched event at 00:51. It answers: staged for the Tuesday run as of 00:51. It adds that the order system is in its maintenance window, so the state may have moved since. The person asking now knows exactly how much to trust it.
“What did we ship this customer last month?”
No live tool covers this; the order system has no such operation and nobody is adding one. The knowledge base answers from the nightly export, and the answer carries “as of the 04:30 sync”. A month-old shipping history does not change overnight, so the staleness does no harm here, which is why this question was routed to the copy in the first place.
What’s worth remembering
- Freshness and load on a fragile source pull against each other, and the shape you pick is mostly a decision about which questions are allowed to demand live data.
- A synchronous call through API Gateway to a translating Lambda is the only shape that answers live, so protect it with a usage plan, request validation, a method cache and reserved concurrency rather than assuming the tool-calling loop stays at a sensible rate.
- EventBridge with an SQS buffer and a dead-letter queue decouples availability and change, and its archive-and-replay is how a missed window gets filled without a manual extract.
- AppFlow, DataSync and Transfer Family are the three landing paths into S3 for SaaS records, file shares and partner drops respectively, and a knowledge base over that bucket is the shape a fragile source can survive.
- Webhook handlers must be idempotent on the provider’s delivery id and acknowledge before doing slow work, because every provider retries and out-of-order delivery is normal.
- Give each tool its own freshness contract and make the assistant say as-of when the answer came from a copy, because an unqualified stale answer is the failure mode that gets someone to act on it.