Exam-style
Users upload bundles of scanned documents through a web form. A bundle is around 300MB, a trained model in Amazon SageMaker AI takes about eight minutes to work through one, and bundles arrive a few dozen times a day at unpredictable hours. The person who uploaded one should get a notification when their result is ready rather than waiting on the page. Which way of serving the model fits?
Reveal the answer
B. Asynchronous inference, because it queues each upload, accepts a large payload and a long run, and hands back a result location instead of holding the connection open
A 300MB payload and eight minutes of processing rule out anything that answers inside the request. Asynchronous inference covers both, with payloads up to 1GB and processing times up to one hour. The caller puts the bundle in Amazon S3 and passes the location; SageMaker AI queues the request and returns an output location straight away. The result is written back to S3, and an Amazon SNS notification is published if the endpoint is configured for one. Autoscaling can take the instance count to zero, so idle hours between uploads are not billed. Batch transform is the other option described as not real-time. It scores a dataset already in S3 when you start a job, with instances running only for the length of that job: no per-request trigger, nothing left to call. Serverless inference caps at a 4MB payload and 60 seconds of processing. A real-time endpoint caps at 25MB and 60 seconds too, which no timeout setting changes, and bills for an idle instance around the clock. Building the queue by hand reaches the same shape and leaves the team owning the workers, the retries and the notifications.
Q. 300MB uploads, eight minutes each, a few dozen a day, and the user gets notified when the result lands. Batch or asynchronous?
A. Asynchronous inference. It takes an S3 location, returns one, processes the request from a managed queue, publishes an Amazon SNS notification on completion, and scales to zero between uploads.
Why? Batch transform scores a dataset already in S3 when you start a job, with no per-request trigger and nobody to notify. These bundles arrive as requests, one user at a time. Serverless inference caps at a 4MB payload and 60 seconds of processing, both far short of a bundle. A real-time endpoint caps at 25MB and 60 seconds too, and bills for an instance all day.