The situation
A media company runs about 400 EC2 instances across nine accounts. The oldest launched three years ago; the newest was launched last week from a launch template that copies the oldest. Roughly 70% are m5.2xlarge, not because 70% of the workloads need eight vCPUs and 32 GB, but because that was the size in the template somebody cloned.
Aggregate CPU utilisation across the fleet sits around 11%. Memory is unknown, because the CloudWatch agent was never rolled out and memory is not an EC2 metric. Finance has a figure from an external review claiming AUD$40,000 a month of waste, and the platform team does not trust it because the reviewer had no memory data either.
Three constraints shape the work. Sixty of the instances belong to a licensed video encoder billed per vCPU, so shrinking them saves twice and growing them costs twice. About a hundred are in Auto Scaling groups behind load balancers, where instance size interacts with group size. And there is no CMDB: the only reliable statement about who owns an instance is its tags, which are about 60% complete.
What actually matters
The first thing that matters is that a recommendation without memory data is still a guess, however much arithmetic went into it. CPU utilisation on its own cannot distinguish a workload with headroom from one that is memory-bound and paging, and shrinking the second because the first looked idle produces an outage that sets the whole programme back. Getting memory reported is a prerequisite rather than a refinement.
The second is that at 400 instances this is a pipeline, not a set of decisions. Anything requiring a human judgement per instance will stall around instance forty. The work is to define a rule, apply it mechanically, and reserve human attention for the exceptions the rule flags.
The third is that instances differ in what a mistake costs. Shrinking a stateless web node in an Auto Scaling group behind a health check is nearly free to get wrong, because the group replaces it. Shrinking a licensed encoder changes a bill in two directions and may breach a licence tier. Shrinking a single stateful instance with no replacement means an outage during the resize and a rollback that takes another one. The same recommendation carries different risk depending on what it is attached to.
The fourth is that for a group behind a load balancer, size and count are one decision. Halving the instance size and doubling the group is roughly cost-neutral on paper but changes failure granularity, scaling responsiveness, and per-instance overhead. Right-sizing an Auto Scaling group means asking what total capacity the group needs, then choosing the shape that delivers it, rather than shrinking members in place.
Underneath it, right-sizing is not a project with an end. A fleet that drifted to 11% utilisation once will do it again unless something watches. The output of this work should be a standing mechanism, not a spreadsheet.
What we’ll filter on
- Is there enough evidence to act, including memory, or only CPU?
- What does being wrong cost: an automatic replacement, an outage, or a licence breach?
- Can the change be applied mechanically across many instances, or does each need a decision?
- Is the instance a member of a group, where size and count trade off?
- Is the change reversible, and how quickly?
- Does the fix prevent recurrence, or only correct today’s drift?
The landscape
Compute Optimizer. The recommendation engine. It analyses CloudWatch metrics and returns a specific target instance type per resource, classified as under-provisioned, over-provisioned, or optimised, with a performance risk rating on each option. It covers EC2 instances, Auto Scaling groups, EBS volumes, Lambda and Fargate services. It is opt-in, and enrollable across an organisation from the management account, which is what makes it usable across nine accounts at once. Its recommendations sharpen considerably once memory is reported, and it will tell you when it has insufficient data rather than guessing.
The CloudWatch agent. Memory and disk utilisation are guest-level metrics that EC2 does not publish. The agent reports them as custom metrics, and Compute Optimizer consumes memory when it is available. Rolling the agent out across 400 instances is itself a fleet operation, and Systems Manager Distributor plus State Manager is the mechanism: an association that installs and configures the agent on anything matching a tag, including instances launched later.
Trusted Advisor. Cost checks flag idle and underutilised instances, low-utilisation EBS volumes, unassociated Elastic IPs and idle load balancers. It answers “where is there waste” rather than “what size should this be”, so it complements Compute Optimizer rather than competing with it. The full check set needs a Business or Enterprise Support plan.
Cost Explorer right-sizing recommendations. Cost Explorer produces its own right-sizing recommendations with the saving attached in currency, which is the form finance wants. It is the same underlying idea presented for a different reader, and it is useful for building the business case rather than for driving the change.
Systems Manager Automation. The mechanism for applying hundreds of changes without hundreds of console sessions. A runbook stops an instance, changes the instance type, starts it, and verifies it, with rate control so the fleet rolls in batches and an error threshold that halts the run when failures exceed a limit. The AWS-published runbook library covers resizing directly.
Auto Scaling group refresh. For group members, the change is to the launch template rather than the instances, followed by an instance refresh that replaces members in batches while respecting a minimum healthy percentage. Nothing is resized in place; the group cycles onto the new shape under a health check, which is what makes group members the safest population to start with.
Attribute-based instance type selection. For groups, requesting capacity by attributes (vCPUs, memory, and constraints such as processor family) rather than by named instance type lets the group draw from any type that qualifies. It reduces the value of getting the exact type right and improves Spot availability where the group can use it.
Savings Plans and Reserved Instances. Compute Savings Plans commit to a dollar-per-hour of compute and apply across EC2, Lambda and Fargate regardless of instance family, region or size, which makes them the commitment instrument that survives a right-sizing programme. Standard Reserved Instances lock to an instance family and are worth less when the fleet is about to change shape.
Graviton. Where a workload runs on open-source software or a language runtime rather than a compiled x86 binary, moving to a Graviton family reduces the rate independently of size. It is a different axis from right-sizing and can be taken in the same change.
Evaluation
Side by side
| Tool or lever | Answers | Needs memory data | Applies at scale | Risk if wrong |
|---|---|---|---|---|
| Compute Optimizer | What size should this be | Sharper with it | Recommendation only | Under-sizing |
| CloudWatch agent | What is memory actually doing | It provides it | Via State Manager | None |
| Trusted Advisor | Where is there waste at all | ✗ | Report only | None |
| Cost Explorer recommendations | What is the saving in dollars | ✗ | Report only | None |
| SSM Automation | Applying a resize to many instances | n/a | ✓ with rate control | Outage per instance |
| ASG launch template + refresh | Resizing group members | n/a | ✓ in batches | Absorbed by health checks |
| Attribute-based selection | Avoiding the exact-type decision | n/a | ✓ | Mixed families |
| Compute Savings Plans | Discounting whatever survives | n/a | ✓ | Commitment |
The table splits into three jobs that get confused with each other: finding waste, deciding the target size, and applying the change safely. Trusted Advisor and Cost Explorer do the first, Compute Optimizer does the second, and Systems Manager and Auto Scaling do the third. A scenario naming one of them as the whole answer is usually wrong.
The solution
Instrument first, then group members, then singletons, then commit. The order is chosen by what a mistake costs, cheapest first, and by what has to be true before a recommendation can be trusted.
Nothing worth acting on exists until memory is reported, so the first change is the CloudWatch agent, rolled out with Systems Manager Distributor and held in place by a State Manager association targeting a tag. The association reapplies, so an instance launched next month arrives with the agent rather than becoming a new blind spot. Enrol Compute Optimizer at the organisation level at the same time, and then leave both alone for a fortnight, because a recommendation built on three days of data is not better than the guess it replaces.
While that accrues, use Trusted Advisor and Cost Explorer to establish the number. This is the step that answers finance without committing to anything: idle instances, unattached volumes, orphaned Elastic IPs and idle load balancers are waste that needs no sizing decision at all, and deleting them is both the fastest saving and a useful demonstration that the programme is real.
Then take the hundred Auto Scaling group members, because they are the population where being wrong is cheapest. Change the launch template rather than the instances, and run an instance refresh with a minimum healthy percentage set so the group stays in service while it cycles. Where a group’s workload allows it, switch to attribute-based instance type selection at the same time, which turns “which type” into “how much vCPU and memory” and stops the next person cloning a named type into a template. For each group, decide total capacity first and shape second: two smaller instances instead of one larger one gives finer failure granularity and faster scaling steps, at the cost of more per-instance overhead.
Singletons come next, through Systems Manager Automation with rate control. Take Compute Optimizer’s recommendation, filter out anything it rates as high performance risk, filter out anything without an owner tag until an owner is found, and hold the sixty licensed encoders out of the automated run entirely. Roll in batches with an error threshold, so a batch that fails stops the run rather than continuing into the rest of the fleet. Each instance takes a stop and start, so this is the population that needs a window and a communication.
The encoders get handled by hand, because the licence is billed per vCPU and the arithmetic runs both ways. Shrinking them reduces both the instance cost and the licence cost, which makes them the highest-value instances in the fleet to get right and the worst to get wrong. Measure, propose, and have somebody who owns the licence agreement sign it off.
Buy Compute Savings Plans last, once the fleet has settled, and choose Savings Plans over Standard Reserved Instances precisely because the fleet has just proved it changes shape. A commitment that survives a family change is worth more here than a slightly deeper discount that does not.
Why not act on CPU alone and move faster. It is the tempting shortcut and it produces the outage that ends the programme. A workload at 10% CPU and 90% memory looks identical to one at 10% CPU and 20% memory in the only metric available by default.
Why not resize group members in place. It works, and it throws away the safest property those instances have. Changing the launch template and refreshing means every replacement passes a health check before the next one starts.
Worked example
The agent rollout reaches 380 of the 400 instances in the first pass. The 20 failures are all in one account with a missing instance profile, which is a five-minute fix and a useful early signal that the association is doing its job.
Two weeks later Compute Optimizer has recommendations for 341 instances and reports insufficient data on the rest. Of the 341, it rates 244 over-provisioned, 12 under-provisioned, and 85 optimised. The 12 under-provisioned are the interesting ones: three are memory-bound at above 90%, which no CPU-only review would have caught, and one of those is the instance a team has been complaining about for a year.
The Auto Scaling groups go first. Six groups move from m5.2xlarge to m6g.xlarge, halving the size and taking Graviton in the same launch template change, with the group minimum raised from 4 to 6 to hold total capacity roughly constant. The instance refresh runs at 50% minimum healthy and takes about forty minutes per group with nothing user-visible. That alone is AUD$14,000 a month.
The singleton run covers 168 instances across three Sunday windows, at 20 instances a batch with a failure threshold of 2. One batch halts on an instance that fails its post-resize status check; it turns out to have a hardcoded reference to an ephemeral device that does not exist on the smaller type. The halt is the mechanism working: one instance is investigated instead of the remaining 148 being resized behind it.
The encoders end up shrinking from 16 vCPU to 8 after a fortnight of measurement shows peak utilisation at 45%, which halves both the instance bill and the licence, and is signed off by the person whose name is on the licence rather than by the platform team.
Aggregate CPU utilisation across the fleet lands near 34%, which is a healthier number than 11% and a long way from the 70% that would mean no headroom. The three under-provisioned instances get bigger, which is not a saving and is the reason to trust the rest.
What’s worth remembering
- Memory is not an EC2 metric, so a right-sizing recommendation built without the CloudWatch agent cannot tell a workload with headroom from one that is memory-bound; install the agent before trusting anything.
- Compute Optimizer says what size a resource should be; Trusted Advisor and Cost Explorer say where waste is. They answer different questions and a complete answer usually needs both.
- Order the fleet by what a mistake costs: Auto Scaling group members first, because a health check absorbs the error, then singletons, then anything with a licence attached to it.
- Resize group members by changing the launch template and running an instance refresh, not by resizing in place, and treat size and group count as one decision about total capacity.
- Apply bulk changes through Systems Manager Automation with rate control and an error threshold, so a bad batch halts the run instead of propagating through the fleet.
- Commit last and commit flexibly: Compute Savings Plans apply across families, sizes and regions, which is what you want immediately after proving the fleet’s shape can change.