Exam-style
A regional insurer wants plain-language summaries of claim notes inside its claims tool. The team has compared several models and the one that reads the notes best is proprietary: the provider offers it through an API and publishes no weights. There are four application developers and nobody who runs ML infrastructure. The work arrives in two bursts a day and adds up to a few hundred summaries, and the team expects to move to a better model within the year. Which route fits?
Reveal the answer
C. Call the proprietary model through the Amazon Bedrock API, where AWS owns the instances and the scaling and the team pays for the tokens it sends and receives
Two decisions are tangled together in this scenario, and separating them settles it. The first is where the model comes from. There are three sources: open source pre-trained models, whose weights anyone can download; proprietary models, reached through the provider’s API; and custom models trained on your own data. The second is how the model gets served. There are two methods: a managed API service, where AWS owns the instances, the scaling and the patching and you pay per call, and a self-hosted API, where the model runs on infrastructure in your account and the instance type, the autoscaling policy, the network placement and the patching are all yours. What is easy to miss is that the first decision constrains the second. A self-hosted API needs an artefact to put on the instance, and the model the team has chosen has none to obtain, so the only serving method left open is a managed API service. That is what Amazon Bedrock is: the model behind an API call, no endpoint to keep warm through the hours between the morning and afternoon bursts, and a model swap later that is a parameter change rather than a redeployment. Deploying from SageMaker JumpStart fails on availability before it fails on anything else, since JumpStart deploys models whose weights the team can get, and a real-time endpoint then bills by the hour it exists rather than by the few hundred summaries that reach it. Fine-tuning an open-weight model swaps the source decision to satisfy an infrastructure preference nobody stated, and hands four application developers a serving layer with nobody to run it. Training from scratch buys weights the insurer has no reason to own, at the cost of a labelled corpus, a training bill and months before the first summary. A purpose-built AI service is a finished capability with a fixed API: Amazon Comprehend extracts entities, key phrases and sentiment, which is text analysis of a different kind and not the summary the claims handler is going to read. Carry the default away from this one. A managed API service is the faster and cheaper route until a requirement about instance sizing, network placement or model weights you hold yourself asks for something else.
Q. A proprietary model with no published weights, nobody to run ML infrastructure, and a few hundred summaries a day in two bursts. Which route?
A. The managed API service: call the model through Amazon Bedrock and pay per token, with no endpoint of your own to keep warm through the quiet hours between bursts.
Why? Keep the two decisions apart. Where the model comes from has three answers: open source pre-trained models, proprietary models behind a provider’s API, and custom models trained on your own data. How it gets served has two: a managed API service, where AWS owns the instances and the scaling, or a self-hosted API, where you own all of it. The first decision narrows the second, because you can only self-host weights you can obtain, and a proprietary model with none published leaves one serving method standing. Take on a self-hosted API when something specific asks for it, such as an instance type you have to name or traffic that has to be served from inside your own VPC. Nothing here does, and an endpoint sized for the busiest ten minutes bills for the other twenty-three hours as well.