Exam-style
A regional insurer wants plain-language summaries of claim notes inside its claims tool. The model that reads the notes best is proprietary: the provider offers it only through an API and publishes no weights. There are four application developers and nobody who runs ML infrastructure. The work arrives in two bursts a day and adds up to a few hundred summaries, and the team expects to move to a better model within the year. Which route fits?
Reveal the answer
C. Call the proprietary model through the Amazon Bedrock API, where AWS runs the instances and the team is billed per input and output token
Two decisions are tangled here. Where the model comes from has three answers: open weight pre-trained models, proprietary models reached through a provider’s API, and custom models trained on your own data. How it gets served has two: a managed API service, where AWS runs the instances and the scaling, or a self-hosted API, where the instance type, the scaling and the patching are yours. The first decision narrows the second, because self-hosting needs an artefact for the instance. The JumpStart catalogue carries proprietary models too, but not this one, and this provider publishes nothing to deploy. Amazon Bedrock is the managed route: no endpoint to keep warm between the bursts, and a later model swap is a change of modelId rather than a redeployment. Fine-tuning and training from scratch both hand four application developers a serving layer nobody can run. Amazon Comprehend detects entities, key phrases and sentiment; it does not write summaries.
Q. A proprietary model with no published weights, nobody to run ML infrastructure, and a few hundred summaries a day in two bursts. Which route?
A. The managed API service: call the model through Amazon Bedrock and be billed per input and output token, with no endpoint of your own to keep warm through the quiet hours between bursts.
Why? Keep the two decisions apart. The model’s source has three answers: open weight pre-trained models, proprietary models behind a provider’s API, and custom models trained on your own data. Serving has two: a managed API service, where AWS runs the instances and the scaling, or a self-hosted API, where you run all of it. The first decision narrows the second, because you can only self-host weights you can obtain. Choose a self-hosted API when something specific requires it, such as an instance type you have to name or traffic that has to stay inside your own VPC. Nothing here does. A real-time endpoint sized for the busiest ten minutes bills for its instances the rest of the day too. Scaling one to zero is possible, but it needs inference components, a step scaling policy and a CloudWatch alarm, and the first request after a scale-in errors while an instance starts.