The Lab Reaper

A safety net for the hands-on labs. Every lab deploys real AWS resources into your own account and expects you to tear them down when you finish. The reaper is what catches the session you walked away from: a small scheduled Lambda that deletes lab infrastructure once it passes an expiry, so a forgotten stack turns into a deleted stack instead of a surprise bill.

It is on and enforcing by default the moment you deploy it, and it lives outside the labs: one stack you deploy once, not something baked into each lab.

What it deletes, and what it will never touch

The lab deploy scripts tag everything they create with two tags:

lab-reaper:managed    = true
lab-reaper:expires-at = <unix epoch seconds, UTC>

Every deploy.sh sets expires-at to 24 hours out and refreshes it on each run. The reaper deletes a resource only when both tags are present, managed is exactly true, and expires-at is a time in the past. A missing or unreadable expires-at means “leave it alone”, never “delete it”.

That tag format is the only thing the reaper acts on. It does not match on stack names, prefixes, or resource types, so nothing else in your account is a candidate, whatever it is called.

Each run sweeps, per region:

  1. CloudFormation stacks (labs 01-13). Before deleting a stack it empties any S3 bucket the stack owns, because CloudFormation will not delete a non-empty bucket. It empties a bucket only after re-checking that the bucket itself carries lab-reaper:managed=true, so an untagged bucket is never touched even if it somehow sits inside a lab stack.
  2. Bedrock serving capacity (lab 12): provisioned throughputs and custom model deployments, billed by the hour, created outside CloudFormation.
  3. Bedrock custom models (lab 12): fine-tuned models, billed monthly for storage, also outside CloudFormation.

Deploy it

Once per account, in the region you run labs in:

cd lab-reaper
./scripts/deploy.sh

Options (environment variables):

# Sweep more than one region (add us-west-2 for lab 13 or west-coast fine-tuning)
REAP_REGIONS=us-east-1,us-west-2 ./scripts/deploy.sh

# Watch before you enforce: log what would be deleted, delete nothing
DRY_RUN=true ./scripts/deploy.sh

# Change the cadence (any EventBridge schedule expression)
SCHEDULE='rate(2 hours)' ./scripts/deploy.sh

REGION (via AWS_REGION) is where the reaper Lambda runs; REAP_REGIONS is the list of regions it sweeps. They default to the same us-east-1 the labs default to. The reaper is regional: to clean stacks in a region, that region must be in REAP_REGIONS.

Trigger a run immediately instead of waiting for the schedule (it honours DRY_RUN, and prints what it did):

aws lambda invoke --function-name lab-reaper --region us-east-1 /dev/stdout

Keep something alive longer

Re-running any lab’s own scripts/deploy.sh resets that stack’s clock to 24 hours out. To extend without redeploying (handy for lab 11’s Knowledge Base, which is slow to rebuild), re-tag the stack in place:

./scripts/extend.sh genai-lab-11        # +24h from now
./scripts/extend.sh genai-lab-11 72     # +72h from now

Change the default window for a fresh deploy with LAB_TTL_HOURS:

LAB_TTL_HOURS=48 ./scripts/deploy.sh    # run inside a lab dir

Turn it off

./scripts/teardown.sh

This removes only the reaper’s own Lambda, role, and schedule. Your lab stacks are untouched; they keep their expiry tags, but nothing acts on them until a reaper is deployed again. You are back to manual teardown.sh per lab.

The trade-off to know about

The reaper’s execution role holds broad delete permissions, because CloudFormation deletes a stack’s resources using that role, and the labs create IAM roles, Lambda functions, DynamoDB tables, S3 buckets, Bedrock guardrails and knowledge bases, and S3 Vectors stores. That breadth is unavoidable for an autonomous janitor. Two guardrails keep it pointed only at lab infrastructure:

The reaper stack is deliberately left untagged, so it can never reap itself.