Lab: Immutable Backups With S3 Object Lock

This is a lab in the SA Pro hands-on track. The reference posts argue the decisions; this stands one up. It pairs with the post on ransomware-proof backups across an organisation, which walks the whole picture: accounts, Regions, and where the immutable copy sits. Here we build the smallest honest slice of it, one source bucket and one guarded destination in a single stack. The full lab is in lab-sa-pro-03-immutable-backup.zip.

If this is your first lab, do the one-time, once-per-account setup first: run the zip’s preflight.sh to confirm your account is ready, then deploy the lab reaper, a standing backstop that auto-deletes any lab you forget to tear down after 24 hours. Every SA Pro lab tags its stack for the reaper on deploy.

The scenario

A backup that lives in the same account and Region as the thing it protects shares that thing’s fate. An attacker who compromises the account can delete the data, delete the backups, and then delete the versions of the backups, because every one of those deletes is a call the compromised account is allowed to make. What earns the name backup is a copy the account cannot destroy even when it is fully owned: a copy where the delete is simply not permitted, for anyone, until a retention clock runs out.

S3 gives you both halves. Replication makes the second copy automatically on every write, with no batch job to forget. Object Lock makes that copy immutable, so a version under retention cannot be overwritten or deleted until the lock expires. The lazy answer is a nightly copy script and a bucket policy; both are things the same account can change or skip. The answer that scores on a Professional paper is replication into an Object-Lock bucket, ideally in another account, so the copy sits outside the blast radius of the credentials that wrote it.

What you’re given

CloudFormation builds the destination bucket with versioning and S3 Object Lock, and the source bucket with versioning. The destination is already guarded: its Object Lock configuration sets a default retention on every new version.

DestinationBucket:
  Type: AWS::S3::Bucket
  Properties:
    VersioningConfiguration:
      Status: Enabled
    ObjectLockEnabled: true
    ObjectLockConfiguration:
      ObjectLockEnabled: Enabled
      Rule:
        DefaultRetention:
          Mode: GOVERNANCE
          Days: 1

Two things about that block are worth pinning down. Object Lock can only be turned on when the bucket is created, and it requires versioning, which is why both properties sit together. And the mode is GOVERNANCE, not COMPLIANCE, on purpose: GOVERNANCE lets a caller holding s3:BypassGovernanceRetention delete a locked version by asking for the bypass, which is how the teardown script cleans up in minutes. In production you would use COMPLIANCE, which nobody, not even root, can bypass or shorten until retention expires. That is true immutability, and it is also why a COMPLIANCE bucket cannot be torn down on demand.

The gap is the source bucket. As shipped it has versioning but no replication rule, so writes land in the source and never reach the guarded destination.

Your task

Add two things to src/template.yaml. First, an IAM role S3 assumes to replicate, trusting s3.amazonaws.com and carrying the standard replication permissions: read the replication config and list the source, read each object version on the source, and write the replicated object, delete marker, and tags on the destination. Second, a replication rule on the source bucket:

SourceBucket:
  Type: AWS::S3::Bucket
  Properties:
    VersioningConfiguration:
      Status: Enabled
    ReplicationConfiguration:
      Role: !GetAtt ReplicationRole.Arn
      Rules:
        - Status: Enabled
          Priority: 0
          DeleteMarkerReplication:
            Status: Disabled
          Filter:
            Prefix: ''
          Destination:
            Bucket: !GetAtt DestinationBucket.Arn

The empty-prefix Filter matches every object. DeleteMarkerReplication is disabled so a delete on the source cannot propagate a delete marker to the guarded copy: the destination keeps what the source loses. Encryption is left at the bucket default, SSE-S3, so there is no KMS key to grant the role.

One detail decides whether the stack even builds. The source’s replication config references the role, and the role’s policy needs the source bucket’s ARN. Write that ARN with !GetAtt SourceBucket.Arn and you get a dependency cycle: source needs role, role needs source, and CloudFormation refuses. Build the source ARN from a !Sub string using the same name the bucket uses, and the cycle disappears because the role no longer points back at the source resource.

Run it

./scripts/deploy.sh          # deploys src/template.yaml
./scripts/test.sh            # writes an object, checks replication and the lock
./scripts/teardown.sh        # empties both buckets and deletes the stack

test.sh writes one object to the source, polls the destination for up to a minute for the replicated copy, then takes that copy’s version and tries to delete it without a bypass. With the starter template nothing replicates, so the test stops and tells you the rule is missing. With the role and rule in place it prints:

PASS: the object replicated to the guarded bucket, and Object Lock refused to delete the protected version.

If it fails

• Nothing replicates. The source bucket has no replication rule, or the role is missing or lacks the read permissions on the source. Replication is asynchronous, so give it the full minute; if it still never arrives, the rule is the thing to check.
• The stack will not build (circular dependency). The role’s policy is referencing the source bucket with !GetAtt SourceBucket.Arn. Build the source ARN from a !Sub string instead so the role does not depend on the source resource.
• The delete succeeds. The object replicated but Object Lock is not guarding the destination. Confirm the destination has ObjectLockEnabled: true, an ObjectLockConfiguration with a default retention, and versioning on.

Reveal the solution

SRC=solution ./scripts/deploy.sh && ./scripts/test.sh

What you just learned

• A backup inside the same account and Region as its source shares that source’s blast radius. Separation and immutability are what make it a backup rather than a copy.
• Replication makes the second copy on every write, no schedule to miss. The role is assumed by S3 itself; its permissions are read-on-source and write-on-destination.
• Object Lock protects a version. GOVERNANCE can be bypassed with a permission and an explicit flag; COMPLIANCE cannot be bypassed by anyone until retention expires.
• Object Lock is a create-time, versioning-required setting. You cannot switch it on for a bucket that already exists.
• Same-Region here and cross-Region or cross-account in production are the same configuration. The rule points at a destination bucket ARN; where that bucket lives is a deployment choice, not a code change.

Next

The rest of the SA Pro lab track is in the README.