SAA Lab 02 - Bake an image and replace a fleet without logging in

Scaffold: 4/5. The VPC, the Image Builder component, recipe, build infrastructure and image, the launch template and the Auto Scaling group are all built and wired. What is missing is the instance refresh: the configuration that takes the fleet from one image to the next.

The scenario

Software reaches a fleet one of two ways. Either something logs in to a running instance and changes it, or the instance is replaced by one launched from a new image. The first leaves you with servers whose contents are the sum of every change anyone ever made to them. The second leaves you with servers whose contents are the image, and an image you can point at.

This lab builds the second. An Image Builder recipe bakes a marker file into an AMI, a launch template names that AMI, and an Auto Scaling group runs two instances from it. Then you edit one instance by hand, bake a second image, and run an instance refresh. The hand edit disappears with the instance that carried it.

The requirement

Write src/refresh.json: the input to aws autoscaling start-instance-refresh that replaces every instance not already on the new launch template version, keeps the fleet in service while it runs, and undoes itself if the replacements fail to come up.

test.sh substitutes AUTO_SCALING_GROUP, LAUNCH_TEMPLATE_ID and LAUNCH_TEMPLATE_VERSION from the stack outputs before it calls the API, so use those placeholders verbatim.

Run it

./scripts/deploy.sh          # bakes image one and starts the fleet
./scripts/test.sh            # drifts an instance, bakes image two, refreshes
./scripts/teardown.sh        # deletes the stack, the AMIs and their snapshots

Two Image Builder builds run, at roughly fifteen minutes each, so set aside about forty minutes end to end. Most of it is waiting.

Then break it on purpose

Change the launch template version in your refresh.json from LAUNCH_TEMPLATE_VERSION to "$Default" and run test.sh again. The API refuses to start the refresh, because auto rollback needs a specific numbered version to roll back to. Then take SkipMatching out and watch the refresh replace instances it had no reason to touch.

Reveal the solution

SRC=solution ./scripts/test.sh

Cost

Six t3.micro instances over the life of the lab (two build instances, two fleet instances, and two more briefly during the refresh), two AMIs and their EBS snapshots. Small, and it bills while it exists. Tear down when you are finished; the lab reaper is a backstop, not a plan.