Exam-style
A distributor has published a target to halve its absolute emissions by 2032, audited every year. Every new system now has to state its expected energy contribution before sign-off. The team wants to add a summarisation feature: 20,000 documents a week, about 900 words in and 120 words out, with quiet nights and quiet weekends. The catalogue offers several sizes of the same model family through Amazon Bedrock. Which approach best reflects responsible practices to select a model?
Reveal the answer
C. Evaluate the smallest model in the family first, and if it clears the quality bar, serve it through on-demand invocation
Two numbers drive the energy a generative feature draws, and both are settled at design time. The first is model size. Producing a token means reading the model’s parameters and doing arithmetic with them, so a model with a tenth as many parameters does roughly a tenth as much work per token. The second is how many tokens pass through, counting the prompt as well as the response. Choosing the smallest model that clears the evaluation bar cuts the first number. Which size that is cannot be known without measuring, so the evaluation comes before the choice. On-demand invocation handles the rest. It shares a pool across many customers, so the quiet nights and weekends are filled by somebody else’s traffic rather than by idle hardware kept powered and cooled. That is what rules out Provisioned Throughput here. Reserved capacity is efficient when it stays busy, and this one would sit cold most of the week while still drawing power. Pre-training from scratch is the most expensive answer here by orders of magnitude. Thousands of accelerators run for weeks, and what comes out has read far less English than the model already sitting in the catalogue. Reusing a pre-trained model is the single largest reduction available, and the team gets it by using Bedrock at all. Offsets are an accounting entry, not a reduction. Reach for them instead of dropping a model size and the compute stays exactly where it was. Sustainability guidance asks you to reduce first, then account for what is left. The managed-service answer misreads who owns what. The AWS shared responsibility model has a sustainability version, written down in the sustainability pillar of the AWS Well-Architected Framework. AWS owns data-centre efficiency, cooling, hardware refresh and the energy mix of each Region. You own the demand you place on it: how large a model you invoke, how many tokens you send, how much capacity you hold idle, which Region you run in. Nothing on the customer side of that line moves because the hardware is somebody else’s.
Q. A published emissions target, a sign-off form asking for the energy cost, and a summarisation feature with quiet nights. Which model choice?
A. Measure the smallest model in the family against the evaluation set, and if it clears the bar, serve it on demand.
Why? Inference energy scales with model size and with tokens processed. The smallest model that does the job, on the shortest prompt that works, cuts more than anything else available here. Reserved capacity draws power whether requests arrive or not, which suits a busy workload and not this one. Pre-training from scratch costs orders of magnitude more than reusing a model somebody has already trained, and offsets record compute rather than reducing it. Under the shared responsibility model AWS owns data-centre efficiency and the Region’s energy mix; you own how much compute you ask for, which is what the sustainability pillar of the Well-Architected Framework asks you to minimise. Environmental considerations sit alongside accuracy, cost and latency. A smaller model that gets the summary wrong has saved nothing.