The situation
An insurance quoting platform has passed its annual review every year and failed this one. The reviewer used Security Hub with the Foundational Security Best Practices standard enabled for the first time, and the result is 140 findings across three accounts: 12 critical, 31 high, and the rest medium and low.
The system is a fleet of EC2 instances behind an Application Load Balancer, an RDS PostgreSQL database, an S3 bucket of policy documents, and a set of Lambda functions doing overnight reconciliation. It processes about 8,000 quotes a day and the business will not accept downtime during business hours.
Reading the findings, the pattern is familiar. Instances have IMDSv1 enabled. The S3 bucket has no explicit encryption configuration and no public access block. Several IAM roles carry * on actions. Security groups allow 0.0.0.0/0 on port 22. RDS is unencrypted at rest. There is no organisation trail. Half a dozen instances are running an OS version past its patch baseline.
The team has one engineer for six weeks.
What actually matters
The first thing that matters is that severity labels rank findings by category, not by exploitability in this environment. A critical finding on a resource nothing can reach is less urgent than a medium finding on the internet-facing path. The reviewer’s ranking is a starting point and the team’s own ranking is the one to work from.
The second is that some of these controls apply from outside the application and some require touching it. Turning on the S3 public access block changes nothing about the application. Encrypting an unencrypted RDS instance requires a snapshot, a restore and an endpoint change, which is a window. Sorting the findings by where the change lands is what makes six weeks feasible.
The third is that a whole class of these findings should be fixed once at the organisation level rather than 140 times at the resource level. Missing S3 public access blocks and missing default encryption are account settings. Open SSH from anywhere is preventable by policy. Doing this at the resource level fixes today’s findings and does nothing about next quarter’s.
The fourth is that a finding closed by an exception is a legitimate outcome, and the exception needs to be recorded where the next reviewer will find it. A workload with 140 findings will not end at zero, and pretending otherwise produces either an exhausted engineer or a suppressed finding nobody can explain in a year.
Underneath all of it, the review found these because somebody enabled a standard. The lasting improvement is that the standard stays enabled and the findings get triaged continuously, rather than accumulating for twelve months until the next review.
What we’ll filter on
- Is this reachable from the internet, or only from inside a boundary?
- Does fixing it require touching the application, a window, or nothing?
- Can it be fixed once at the organisation level instead of per resource?
- Does it prevent recurrence, or only clear today’s finding?
- If it cannot be fixed in six weeks, can it be compensated for?
- Who accepts the risk if it is deferred?
The landscape
Security Hub. Aggregates findings from its own standards and from Inspector, GuardDuty, Macie, IAM Access Analyzer and others, into one normalised view with a severity per finding and a security score per standard. It is the triage surface rather than a scanner in its own right. Findings can be suppressed with a reason, which is the mechanism for recording an accepted risk.
Amazon Inspector. Continuous vulnerability scanning of EC2 instances, container images in ECR and Lambda functions, producing CVE findings with an exploitability-aware score. It is what turns “running an old OS” into a specific list of packages and known exploits, which is what makes patching arguable rather than open-ended.
IAM Access Analyzer. Two capabilities worth separating. External access analysis finds resources shared outside a defined zone of trust, which is the fastest way to answer “is anything actually exposed”. Unused access analysis finds roles, users and permissions nobody has exercised, which is what turns a * policy into a specific, evidence-backed replacement. It also generates policies from CloudTrail history.
Service control policies. Organisation-level prevention. An SCP denying the actions that create a class of finding stops it recurring in every account at once, and no account administrator can override it. It prevents rather than detects, and it cannot grant anything.
AWS Config with conformance packs. Continuous evaluation of resource configuration against rules, with remediation actions attached through Systems Manager Automation. Where an SCP prevents, Config detects and can correct. A conformance pack applies a whole baseline across an organisation as a unit.
S3 account-level settings. Block Public Access can be set at the account level, above every bucket, and default encryption applies to new objects without touching the application. Both are configuration changes with no application impact.
IMDSv2 enforcement. The instance metadata service’s session-oriented version defeats the SSRF-to-credential-theft path that IMDSv1 allows. It can be required per instance, and set as the default for new instances at the account level. Older SDKs and hardcoded metadata calls are the compatibility risk, and the metrics that show whether IMDSv1 is still being used are what makes the change safe.
RDS encryption at rest. Cannot be enabled in place. The path is a snapshot, a copy of the snapshot with encryption enabled, a restore, and a cutover, which means a window and an endpoint change. It is the most disruptive item on this list and one of the easiest to defer with a compensating control if the data classification allows.
Patch Manager. Baselines, maintenance windows, and compliance reporting for OS patching, driven from Systems Manager. It converts “six instances are behind” into a recurring process with a compliance number.
Evaluation
Side by side
| Finding class | Reachable from internet | Fix lands | Window | Preventable at org level |
|---|---|---|---|---|
| Security group 0.0.0.0/0 on 22 | ✓ | Security group | ✗ | ✓ SCP or Config |
| No S3 public access block | ✓ potentially | Account setting | ✗ | ✓ account-level |
| IMDSv1 enabled | Via SSRF | Instance metadata option | ✗ rolling | ✓ account default |
IAM roles with * |
Indirect | Policy rewrite | ✗ | Partly |
| No organisation trail | ✗ | Organisation setting | ✗ | ✓ |
| Unpatched OS | ✓ if exposed | Patch Manager | ✓ rolling | ✓ Config rule |
| RDS unencrypted at rest | ✗ | Snapshot, restore, cutover | ✓ | ✓ for new instances |
| S3 default encryption missing | ✗ | Bucket setting | ✗ | ✓ account-level |
The column that reorders the work is the first one. Four classes are reachable or reachable-adjacent, and they are the first fortnight regardless of severity label. The RDS finding carries a high score and sits at the bottom of the exploitability ranking, because reaching the data at rest requires access to the storage layer that an attacker on the internet does not have. That is the trade to state explicitly rather than to leave implied.
The solution
Rank by reachability, fix the exposed path first, prevent the recurring classes at the organisation level, and write down what gets deferred.
Week one is the internet-facing path. Close the security groups allowing 0.0.0.0/0 on port 22, replacing SSH access with Session Manager, which needs no inbound rule at all and produces a logged session. Run IAM Access Analyzer external access analysis to confirm nothing else is shared outside the account, and set S3 Block Public Access at the account level rather than per bucket. None of this needs a window and all of it reduces what an outsider can reach.
Week two is IMDSv2 and the organisation trail. Enable the metadata service metrics first to see whether anything is still calling IMDSv1, then require IMDSv2 on the instances that show no v1 usage, rolling through the Auto Scaling group by launch template so nothing is modified in place. Set IMDSv2 as the account default so new instances arrive correct. The organisation trail is a single configuration in the management account and closes a class of finding across every account at once.
Weeks three and four are the recurring classes, and this is where the work stops being about 140 findings. Deploy a conformance pack across the organisation covering the rules these findings map to, with remediation attached where the correction is safe to automate. Add SCPs denying the handful of actions that produce the worst classes, so the next account cannot create the problem. From here, a new resource that violates the baseline is either prevented or corrected without anybody reading a report.
Week five is IAM. Use Access Analyzer unused access findings to identify which permissions in the * policies have actually been exercised, and generate replacement policies from CloudTrail history rather than from a guess about what the application needs. This is the item most likely to break something, which is why it comes after the exposed path is closed and why it goes out behind a change with a fast rollback.
Week six is patching, set up as a process rather than an event. Patch baselines, a maintenance window aligned to the existing Sunday slot, and Patch Manager compliance reporting, so “six instances behind” becomes a number that trends rather than a finding that recurs.
RDS encryption does not fit in six weeks and should be scheduled rather than rushed. Record it as an accepted risk with an owner and a date, note the compensating controls (encryption in transit enforced, no public accessibility, access restricted to the application security group, and the underlying storage encrypted at the AWS layer regardless), and suppress the finding in Security Hub with that reason attached so the next reviewer finds the decision rather than the gap.
Why not work top-down by severity. It is what the report invites and it spends the first fortnight on the RDS finding, which is the least reachable item on the list, while an open SSH port stays open.
Why not fix all 140 individually. About 90 of them are instances of a dozen classes. Fixing the class at the organisation level clears the instances and stops the next ones, and fixing the instances clears the report until the next resource is created.
Worked example
The reachability triage takes half a day and reorders everything. Of the 12 critical findings, 3 are on the internet-facing path. Of the 31 high, 11 are. The remaining 129 are internal or preventive.
Closing SSH is the fastest win and produces the only pushback of the whole project, from two engineers who use SSH daily. Session Manager with port forwarding covers both their use cases, and the logged-session property turns out to be something the compliance team had wanted for two years.
The IMDSv2 rollout finds one caller of IMDSv1: a monitoring agent three versions behind, on eleven instances. Upgrading the agent takes a day and the enforcement then rolls cleanly. Without the metrics step it would have been eleven instances losing their monitoring at 2am.
The conformance pack closes 71 findings on deployment, which is the moment the report stops being demoralising. Nine resources fail remediation and need hand-holding, all of them older resources with a dependency the rule did not anticipate.
The IAM work is the slowest and produces the one incident: a generated policy omits an S3 action used only by the quarterly reconciliation, which nobody ran during the observation window. It surfaces in staging, which is why the change went there first, and the lesson recorded is that CloudTrail-derived policies are bounded by the observation period and quarterly jobs need a longer one.
Six weeks in, Security Hub shows 18 open findings against 140. Twelve are accepted risks with owners, dates and compensating controls recorded. Six are genuinely outstanding, all of them medium, all internal. RDS encryption is scheduled for the next quarter’s maintenance weekend, which is a decision with a date rather than a gap in a report.
What’s worth remembering
- Severity labels rank findings by category, not by exploitability in your environment; re-rank by what is reachable from the internet before spending a single day.
- Sort by where the fix lands: account settings and security group changes need no window, and anything needing a snapshot-restore cycle is a scheduled change rather than a six-week item.
- Fix classes at the organisation level rather than instances at the resource level; a conformance pack plus a few SCPs clears most of a report and stops the next resource recreating the problem.
- Prevention and detection are different jobs: an SCP stops the action, and AWS Config finds and corrects it afterwards. A requirement that something can never be created is not a Config answer.
- Enable IMDSv2 with the metadata metrics on first, because the compatibility risk is an old SDK you cannot see until you look for v1 calls.
- An accepted risk with an owner, a date and a compensating control is a legitimate outcome; record it as a suppression with the reason attached so the next reviewer finds the decision rather than the gap.