Bringing Hand-Built Resources Under IaC

The situation

A media streaming company has about 200 resources in its production account that exist because somebody clicked. Fourteen EC2 instances, four RDS databases, three load balancers, nineteen security groups, two dozen S3 buckets, a Route 53 hosted zone with 60 records, several IAM roles, and a scatter of SQS queues and Lambda functions. Some were created in 2022 and nobody who made them still works there.

Everything else the company builds goes through CloudFormation in a pipeline. These 200 are the exception, and they are the exception that causes incidents: a security group rule changed by hand during an outage, never documented, that broke a deployment four months later.

The constraint is that none of it can be recreated. The databases hold production data, the load balancers have addresses in DNS and in partner firewall rules, and the S3 buckets have names that cannot be reused if deleted. Anything that involves a delete-and-recreate cycle is off the table.

What actually matters

The first thing that matters is the difference between importing a resource and recreating one. CloudFormation’s import operation brings an existing resource under a stack’s management without touching it, provided the template describes it accurately. That accuracy requirement is the whole difficulty: a template that says something the resource does not say will either fail the import or, on some resource types, apply the difference.

The second is that not everything is importable. The set of supported resource types is large and not complete, and a resource type outside it has to be managed a different way or left alone. Establishing which of the 200 are importable is a prerequisite rather than something to discover halfway through.

The third is that the template has to be derived from the resource rather than written from memory. Writing what you believe the security group contains, importing, and finding out you were wrong is the failure mode. Generating the template from the live configuration removes the guessing, and the tooling to do that exists.

The fourth is grouping. Importing 200 resources into one stack makes a blast radius nobody wants and a template nobody can review. Importing them into 200 stacks makes an operational mess. The grouping should follow ownership and lifecycle: things that change together and are owned by the same team belong in the same stack.

Underneath all of it, the reason to do this is to stop the hand-edits, and importing alone does not achieve that. A resource in a stack that somebody can still change in the console is a resource that will drift. The control that prevents it is a separate piece of work.

What we’ll filter on

1. Is this resource type importable by CloudFormation?
2. Can the template be generated from the live resource rather than written?
3. What happens if the template and the resource disagree?
4. Does the operation touch the resource, or only the stack’s record of it?
5. What grouping matches ownership and lifecycle?
6. What stops the next hand-edit?

The landscape

CloudFormation resource import. Brings existing resources into a new or existing stack. The template must describe the resource with the properties CloudFormation requires, and each resource needs a DeletionPolicy of Retain during the import. The operation creates a change set of type IMPORT and executing it makes the stack the owner. It does not modify the resource, provided the template matches; where it does not, the import fails validation for most types.

IaC generator. Scans an account for resources and generates a CloudFormation template describing what it finds, along with the import machinery. This removes the hand-writing step that produces most import failures, and it is the difference between this being a two-week job and a two-month one. It produces a starting template rather than a finished one, so review is still required.

Drift detection. After import, drift detection compares the stack’s resources against the template and reports property-level differences. Running it immediately after each import is the confirmation that the template genuinely matches, and it is cheaper than discovering a mismatch during the next update.

CDK import. For teams working in CDK rather than raw templates, cdk import performs the same operation against a synthesised stack. The underlying mechanism is CloudFormation’s, so the same constraints apply.

Terraform import. Named because the exam distinguishes the tools rather than because the company uses it. Terraform imports into state and, since version 1.5, supports declarative import blocks with generated configuration. The idea is the same; the state file is the difference.

Stack policies. Prevent updates to specified resources within a stack, which protects a database from a template change that would replace it. Applied to the stack rather than to the resource, and evaluated during updates.

Service control policies and IAM. The control that stops the hand-edits. An SCP denying console-originated modification of tagged resources, or denying the mutating actions outside a pipeline role, is what makes the import stick. Without it, everything imported is a candidate for drifting straight back out.

AWS Config drift rules. Continuous detection of resources that have changed outside CloudFormation, with notification or remediation. It is the ongoing measurement once the migration is done.

Deletion policies and retain. DeletionPolicy: Retain and UpdateReplacePolicy: Retain are the safety net for anything holding data. During an import they are required; afterwards they are what stops a template mistake destroying a database.

Evaluation

Side by side

Step or tool	Touches the resource	Fails safe	Effort	What it gives you
Hand-written import template	✗ if correct	Mostly	High, error-prone	Full control of the template
IaC generator	✗	✓	Low	An accurate starting template
Import change set	✗	✓ validation	Low	Stack ownership
Drift detection after import	✗	✓	Low	Proof the template matches
Stack policy	✗	✓	Low	Protection from a bad update
SCP on console mutation	✗	✓	Medium, political	The reason it stays imported
Recreate instead of import	✓ destroys	✗	High	A clean template, and an outage

The last row is the option to name and reject explicitly, because it is what a team does when the import path looks fiddly. It produces the tidiest templates and it is unavailable here, and being clear about that early stops the argument recurring at each awkward resource.

The solution

Generate templates from the live resources, import in small groups ordered by risk, prove each import with drift detection, then close the console.

Start with the inventory, because the plan depends on knowing which resources are importable. Run the IaC generator across the account to produce candidate templates, and cross-reference against the supported resource types. Anything unsupported goes on a separate list with a decision attached: managed another way, replaced later, or accepted as out of scope with an owner.

Group the importable resources by ownership and lifecycle rather than by type. All the resources belonging to the ingest service in one stack, the transcoding service in another, shared networking in a third. Twelve to fifteen stacks for 200 resources is a reasonable shape, and it means a template change has a blast radius somebody can reason about.

Order the groups by what an error costs. Start with a group of S3 buckets and IAM roles where a failed import is an inconvenience, and use it to shake out the process: how the generator’s output needs editing, how the change set reads, how long each step takes, what the review looks like. The fourteenth import should be boring, and it will only be boring if the first was cheap.

For each group, the sequence is the same. Generate the template, review it against the live resource, add DeletionPolicy: Retain to everything, create the import change set, execute it, then immediately run drift detection. That last step is the one to insist on: an import that succeeds proves the template was acceptable, and drift detection proves it was accurate. The difference matters for the properties CloudFormation does not validate on import.

Leave the databases until the process is proven, and add stack policies preventing updates to them before importing. A database in a stack is a database a template change can replace, and the retain policies plus a stack policy are what stop a routine update becoming a restore.

Then close the door, which is the part that determines whether any of this lasts. An SCP denying the mutating API calls to tagged production resources except from the pipeline role turns “please use the pipeline” into a constraint. Expect this to be the politically hardest part, because it removes an escape hatch people have used during incidents, and the answer to that objection is a break-glass role with alerting rather than leaving the door open.

Finish with continuous measurement. An AWS Config rule detecting resources modified outside CloudFormation, and scheduled drift detection through EventBridge and Systems Manager Automation, so the next divergence is a notification rather than a surprise during a deployment.

Why not recreate the smaller resources to get cleaner templates. It is tempting for the security groups and queues, where recreation looks cheap. Security group IDs are referenced by other resources, queue URLs are in application configuration, and “cheap to recreate” is usually an untested assumption.

Why not one stack for everything. It imports in one operation and it produces a template of 200 resources where any update risks all of them, which is worse than the state being fixed.

Worked example

The generator finds 211 resources rather than 200, which is the first useful output: eleven nobody knew about, including two Elastic IPs attached to nothing and an RDS snapshot schedule created for a migration that finished in 2023.

Of the 211, 189 are importable. The 22 that are not include a Route 53 health check type not yet supported and several resources belonging to a service that has since been deprecated, which turns out to be a deletion candidate rather than an import one.

The first group, eight S3 buckets and their policies, imports in an afternoon. Drift detection immediately afterwards reports two buckets as MODIFIED: the generator had not captured a lifecycle rule on one and a logging configuration on the other. Neither breaks anything, both are added to the template, and the lesson shapes every subsequent group: the generator output is a draft.

The security group stack is the fiddliest, because nineteen groups reference each other and the ordering matters. It takes two attempts, the first failing validation on a rule referencing a group not yet in the stack.

The databases go last, with stack policies applied first. The import itself is uneventful, which is the intended outcome of having done thirteen groups beforehand.

The SCP takes three weeks of conversation and goes live with a break-glass role that pages when assumed. It is assumed twice in the following quarter, both times legitimately, and both times the change is retrofitted into the template within a day because somebody was notified.

Six months later, drift detection runs weekly and reports nothing most weeks. The incident that started this, a hand-edited security group breaking a deployment months later, cannot recur in the same form: the edit is either impossible or it is visible within a week.

What’s worth remembering

1. Import brings a resource under stack management without modifying it, provided the template matches; the accuracy of that template is the entire difficulty, which is why generating it from the live resource beats writing it.
2. Not every resource type is importable, so establish the supported set before planning, and give the unsupported remainder a decision and an owner rather than discovering it midway.
3. Run drift detection immediately after each import. A successful import proves the template was acceptable; drift detection proves it was accurate, and those are different claims.
4. Group by ownership and lifecycle rather than by resource type, and order the groups so the cheapest failure comes first and the databases come last.
5. DeletionPolicy: Retain is required during import and worth keeping afterwards, and a stack policy is what stops a later template change replacing a database.
6. Importing does not stop hand-edits. An SCP restricting mutation to the pipeline role is what makes the migration durable, and a break-glass role with alerting is the answer to the objection it will provoke.