Exam-style
Users upload bundles of scanned documents through a web form. A bundle is around 300MB, a trained model in Amazon SageMaker AI takes about eight minutes to work through one, and bundles arrive a few dozen times a day at unpredictable hours. The person who uploaded one should get a notification when their result is ready rather than sitting on the page waiting. Which way of serving the model fits?
Reveal the answer
B. Asynchronous inference, because it queues each upload, accepts a large payload and a long run, and hands back a result location instead of holding the connection open
Two things about this workload rule out anything that answers inside the request: a 300MB payload and eight minutes of processing. Asynchronous inference is built for both. The caller puts the bundle in Amazon S3 and sends the location. SageMaker AI returns an output location straight away and the request waits its turn in a managed queue. The result is written back to S3, with a notification published on completion, which is the notification the upload form needs. The endpoint scales to zero once the queue drains, so the quiet hours between uploads cost nothing. Batch transform is the distractor to understand, because it is also described as not real-time. It scores a dataset that is already sitting in S3. Instances are provisioned for the length of the job and torn down afterwards, with no per-request trigger and nothing left to call. Uploads arriving one at a time through a form are requests rather than a dataset. Turning them into a dataset means holding each bundle until a scheduled run, and the user is waiting on a notification instead. Serverless inference is for small payloads and fast responses on intermittent traffic; 300MB and eight minutes are both far outside its ceilings, so the traffic shape never gets a vote. A real-time endpoint with the timeouts raised pays for an idle instance around the clock to serve a few dozen calls. No timeout setting makes a 300MB body and an eight-minute response a sensible synchronous call. Building the queue by hand reaches the right shape and then asks the team to own the queue, the workers, the retries and the notifications that asynchronous inferencing already manages.
Q. 300MB uploads, eight minutes each, a few dozen a day, and the user gets notified when the result lands. Batch or asynchronous?
A. Asynchronous inference. It takes an S3 location, returns one, runs the work through a managed queue, publishes a completion notification, and scales to zero between uploads.
Why? Batch transform scores a dataset that is already in S3 on a schedule, with no per-request trigger and nobody to notify. These bundles arrive as requests, one user at a time. Serverless inferencing is for small, fast payloads and would hit its ceilings long before eight minutes. A real-time endpoint bills for an instance all day, and cannot hold a connection open that long regardless.