Exam-style
A distributor has published a target to halve its absolute emissions by 2032, audited every year, so every new system must state its expected energy contribution before sign-off. The team wants a summarisation feature: 20,000 documents a week, about 900 words in and 120 words out, with quiet nights and weekends. Amazon Bedrock offers several sizes of the same model family. Which approach best reflects responsible practices to select a model?
Reveal the answer
C. Evaluate the smallest model in the family first, and if it clears the quality bar, serve it on demand
Two numbers set the energy this feature draws, and both are fixed at design time: how large the model is, and how many tokens pass through it, prompt included. A model with a tenth of the parameters does roughly a tenth of the arithmetic per token. Evaluate the smallest size first and stop at the one that clears the quality bar; which size that is cannot be known without measuring. The sustainability pillar of the AWS Well-Architected Framework recommends managed services because sharing capacity across a broad customer base raises utilisation, and it asks you to minimise idle resources. On-demand invocation does both. Provisioned Throughput is billed hourly for as long as it exists, and this workload would leave it idle through the quiet nights and weekends. Pre-training from scratch uses orders of magnitude more compute than reusing a trained model. Offsets record the compute rather than reducing it. The pillar also splits responsibility: AWS delivers efficient shared infrastructure and renewable power, and you minimise what your workload requires, including model size, token volume, idle capacity and Region choice. None of that moves to AWS because the hardware does.
Q. A published emissions target, a sign-off form asking for the energy cost, and a summarisation feature with quiet nights. Which model choice?
A. Measure the smallest model in the family against the evaluation set, and if it clears the bar, serve it on demand.
Why? Inference energy scales with model size and with tokens processed. The smallest model that does the job, on the shortest prompt that works, cuts more than anything else available here. Provisioned Throughput is billed hourly whether requests arrive or not, which suits a busy workload and not this one. Pre-training from scratch uses orders of magnitude more compute than reusing a model somebody has already trained, and offsets record compute rather than reducing it. Under the shared responsibility model AWS delivers efficient shared infrastructure and sources renewable power; you minimise the resources your workload requires, which is what the sustainability pillar of the Well-Architected Framework asks of you. Environmental considerations sit alongside accuracy, cost and latency. A smaller model that gets the summary wrong has saved nothing.