ANS Lab 04 - Prove the path before anyone complains
Scaffold: 3/5. Two VPCs on a transit gateway, an application subnet and a
database subnet, a network ACL, a Network Insights path and a Network Access
Scope are all built, and the global network registration is done for you by
deploy.sh. Two gaps are left, both in src/template.yaml. One breaks a flow
that should work. The other opens a door that should not be open. You find each
one with the tool that can see it, and along the way you find out what each
tool cannot see.
The scenario
The application tier in 10.20.0.0/16 has to reach the database tier in 10.21.0.0/16 on TCP 5432, across a transit gateway. Nothing else may reach that database subnet at all. Those are two requirements, and today they are checked by nobody.
AWS gives you three tools that answer questions about a path without sending a packet, and they are not interchangeable:
- Route Analyzer, in Transit Gateway Network Manager, traces a source attachment to a destination attachment through the transit gateway route tables, forward and return. It analyses transit gateway route tables only: not VPC route tables, not security groups, not network ACLs. It does not work with intra-Region peering.
- Reachability Analyzer models the whole virtual path between two resources in a Region, including route tables, security groups and network ACLs, and names the component that blocks it when it is blocked.
- Network Access Analyzer asks the inverse question. You describe access that must not exist, and it returns every path in your account that matches.
Run the same broken network past all three and the gaps in each of them stop being trivia.
What’s provided
src/template.yaml- two VPCs with a subnet, route table and security group each; a transit gateway with default association and propagation enabled and a VPC attachment on each side; at3.microin each subnet; a network ACL on the database subnet; theAWS::EC2::NetworkInsightsPathholding the connectivity requirement; and theAWS::EC2::NetworkInsightsAccessScopeholding the isolation requirement, andTODOmarkers on every line you touch.scripts/deploy.sh- deploys the stack, creates a global network, registers the transit gateway, and waits for the registration to reachAVAILABLE. Route Analyzer needs that registration and does not tell you when it is missing.scripts/test.sh- runs all three tools and exits non-zero when either assertion fails.scripts/teardown.sh- deregisters, deletes the global network, deletes the stack.solution/template.yaml- the complete, correct template.
Network Manager’s API is global and has exactly one endpoint, in us-west-2,
whatever Region your transit gateway is in. The scripts pass --region
us-west-2 on every networkmanager call for that reason.
This lab has a meter running. A transit gateway bills per attachment-hour on each of the two attachments, so deploy it, work through it, and tear it down in one sitting. Each Reachability Analyzer analysis is $0.10; Network Access Analyzer is $0.002 for each network interface it examines. Global networks are free.
Your task
Step 1: run the tools on the shipped stack
Deploy, then run ./scripts/test.sh. Route Analyzer reports CONNECTED in
both directions. Reachability Analyzer fails. Network Access Analyzer fails.
Read all three outputs before changing anything: this is the moment the
difference between the tools is visible, and you only get it once.
Step 2: fix the flow that should work
Reachability Analyzer returns SUBNET_ACL_RESTRICTION and names the network
ACL on the database subnet. Fill in the TODO (step 2) block in
src/template.yaml with the entry that admits the request:
DataNaclInboundPostgres:
Type: AWS::EC2::NetworkAclEntry
Properties:
NetworkAclId: !Ref DataNacl
RuleNumber: 110
Protocol: 6
RuleAction: allow
Egress: false
CidrBlock: 10.20.0.0/16
PortRange: { From: 5432, To: 5432 }
Redeploy and run the test again. The verdict flips to reachable.
Step 3: close the door you did not know was open
Network Access Analyzer’s finding is a full path, component by component: the
internet gateway, the network ACL entry that admitted the traffic, the security
group rule that admitted it, and the network interface at the end. Three
resources in src/template.yaml are marked TODO (step 3), and there is a
fourth line to remove:
DataDefaultRoute, a 0.0.0.0/0 route to the internet gateway;- the
tcp/22 from 0.0.0.0/0rule inDataSecurityGroup; DataNaclInboundSsh;MapPublicIpOnLaunch: trueonDataSubnet.
Any one of those four kills the path. Remove all four, because a finding tells you a path exists, not which link in it was the mistake. Redeploy, run the test, and both assertions pass.
Step 4: break it at the gateway instead
Everything so far broke inside a VPC. Now break it at the transit gateway, with a blackhole route that swallows the database CIDR:
TGW=$(aws cloudformation describe-stacks --stack-name ans-lab-04 \
--query "Stacks[0].Outputs[?OutputKey=='TransitGatewayId'].OutputValue" --output text)
RTB=$(aws ec2 describe-transit-gateway-route-tables \
--filters "Name=transit-gateway-id,Values=$TGW" \
--query 'TransitGatewayRouteTables[0].TransitGatewayRouteTableId' --output text)
aws ec2 create-transit-gateway-route --blackhole \
--transit-gateway-route-table-id "$RTB" --destination-cidr-block 10.21.0.0/16
Run ./scripts/test.sh again. Route Analyzer now returns NOT_CONNECTED with
BLACKHOLE_ROUTE_FOR_DESTINATION_FOUND, which is the most precise answer any
of the three will give you all day. Reachability Analyzer also fails, with a
TGW_RTB_ code naming the transit gateway route table. Network Access Analyzer
still passes, because a blackhole route does not create internet access.
Undo it:
aws ec2 delete-transit-gateway-route \
--transit-gateway-route-table-id "$RTB" --destination-cidr-block 10.21.0.0/16
Step 5: break the half nobody checks
Delete DataNaclOutboundReplies from src/template.yaml and redeploy. The
request still arrives at the database and the reply is dropped on the way out,
so no connection can ever complete.
Run the test. Everything passes. Reachability Analyzer analyses forward traffic only for TCP when the path traverses a transit gateway route table, so the missing egress entry is outside what it looked at. Route Analyzer never reads network ACLs at all. Network Access Analyzer was asked a different question.
Put the entry back before you move on.
Run it
# Defaults: stack ans-lab-04, region ap-southeast-2.
./scripts/deploy.sh # stack, global network, transit gateway registration
./scripts/test.sh # three tools, two assertions
./scripts/teardown.sh # deregister, delete the global network, delete the stack
What success looks like
After steps 2 and 3, ./scripts/test.sh prints the Route Analyzer forward and
return verdicts, then:
PASS the path is reachable
PASS no path from an internet gateway into the database subnet
PASS: both assertions hold. The requirement is reachable and the
database subnet is not.
and exits 0. That exit code is what makes it usable from a pipeline: a connectivity requirement that fails a build is a requirement, and one that lives in a runbook is a hope.
If it fails
- Route Analyzer errors, or returns nothing useful. The registration has
not settled.
deploy.shwaits forAVAILABLE, so if you called the API by hand, checkaws networkmanager get-transit-gateway-registrations --region us-west-2 --global-network-id <id>first. Also check you passed--region us-west-2: the call fails in your own Region because the service has no endpoint there. - Registration FAILED. A transit gateway can be registered in only one
global network, and not every Region supports transit gateways for global
networks. A leftover global network from an earlier run is the usual cause;
teardown.shremoves it. - Reachability Analyzer returns
NO_ROUTE_TO_DESTINATION. A VPC route to the transit gateway did not create. Routes pointing at a transit gateway fail until the VPC has an attachment, which is what theDependsOnin the template is for. - The stack will not delete. Deregister the transit gateway first. A registered gateway cannot be deleted and CloudFormation does not explain the refusal.
Reveal the solution
Deploy the complete reference template without editing anything:
SRC=solution ./scripts/deploy.sh && ./scripts/test.sh
What you just learned
- The three tools have different blind spots, and the blind spots are the reason to know which one you are holding. Route Analyzer sees transit gateway route tables and nothing else. Reachability Analyzer sees one configured path in one Region. Network Access Analyzer sees every path that matches a pattern you said must not exist.
- Route Analyzer needs a global network with the transit gateway registered,
and the API lives only in
us-west-2. Both are prerequisites that fail in ways that do not name themselves. - Network ACLs are stateless and every flow needs an entry in each direction. Over a transit gateway, a forward path analysis for TCP checks the forward direction only, so the missing return entry is the one that gets into production.
- A finding names a path, not a mistake. The internet exposure here was four separate resources, each of which looked defensible on its own.
- An assertion that exits non-zero is a control; an analysis you read in the console is an observation. The same three API calls do either job.
Next
The rest of the Advanced Networking Specialist material is in The Exam Room.