Keeping Tenants Apart in a Shared VPC

The situation

A government services agency runs a shared VPC from a network account, with subnets shared into fourteen workload accounts through Resource Access Manager. The arrangement exists because the agency’s address allocation is small and giving each team its own VPC would exhaust it.

Each team owns its subnets and launches its own resources. The network account owns the VPC, the route tables, the NAT gateways and the network ACLs. Security groups can be created by the participant accounts and are the main control in use.

An assessment has raised three findings. Teams can create security groups referencing any CIDR in the VPC, so nothing prevents one team’s workload reaching another’s database. Two of the fourteen teams handle protected data with a legal requirement that other teams cannot access it. And a shared NAT gateway means every team’s egress traffic mixes, so an investigation cannot attribute a connection to a team without correlating flow logs against ENI ownership.

What actually matters

The first thing that matters is that a VPC is one routing domain. Every subnet in it can reach every other subnet by default, because they share a route table lineage and the local route cannot be removed. Subnets look like boundaries on a diagram and are not, and any design treating “different subnet” as “isolated” is mistaken from the start.

The second is that a shared VPC divides ownership in a specific way that decides what each control can do. The owner account controls route tables, NACLs and the VPC itself; participants control their own resources and security groups. That split determines which controls a tenant can undermine and which they cannot, and it is the fact the exam tests most often about this arrangement.

The third is that the strength of isolation required is a function of who the tenants are. Teams inside one organisation that would not deliberately attack each other need controls that prevent accidents. Teams with a legal separation requirement need controls that a determined insider cannot bypass, which is a different bar and frequently a different architecture.

The fourth is that isolation without attribution is incomplete. If an investigation cannot say which team made a connection, the control set has a gap regardless of how well it restricts. Shared egress through one NAT gateway is the usual cause.

Underneath all of it, some tenants should not be in this VPC at all. The address-space argument that produced the shared VPC is real, and it does not apply equally to a workload with a legal isolation requirement, where a separate VPC and a few addresses is a cheap answer to an expensive problem.

What we’ll filter on

1. Can a tenant remove or weaken this control themselves?
2. Does it prevent accidents, or resist a determined insider?
3. Does it restrict at the network layer, the identity layer, or both?
4. Can an investigation attribute a flow to a tenant?
5. Does it scale as tenants are added?
6. Does the address-space saving still justify the arrangement for this tenant?

The landscape

Security groups in a shared VPC. Participants create and manage security groups for their own resources, and can reference security groups belonging to other participants. This is the flexible, default-permissive control: nothing stops a team writing a group that permits a wide CIDR. It prevents accidents when written well and does not resist a team that writes a bad rule, because they own the rule.

Network ACLs. Owned by the VPC owner account, applied per subnet, stateless, and supporting explicit deny. A participant cannot change them. This makes NACLs the control that expresses “team A’s subnets may not reach team B’s subnets” in a way the tenants cannot undo, and the stateless evaluation and rule-number ordering are the operational cost. Rule limits mean this does not scale to fine-grained per-tenant policy across many tenants.

Route tables. Also owner-controlled. They cannot remove the VPC-local route, so they cannot isolate subnets within a VPC from each other. They control egress paths, which is how per-tenant NAT is arranged. Worth knowing precisely because the exam likes the misconception that a route table can isolate two subnets in the same VPC.

Service control policies and IAM. Prevent a participant from creating a resource or a rule in the first place. An SCP can deny the creation of security group rules with overly wide sources, or deny particular actions entirely, and no account administrator can override it. This is the layer that constrains what a tenant can configure, as opposed to what their packets can reach.

Separate VPCs with PrivateLink. The strong isolation answer. Different VPCs share no routing domain, and a tenant that needs one specific service from another gets a PrivateLink endpoint to exactly that service. It costs addresses, though far fewer than a full VPC per tenant if the VPCs are small, and it removes the entire class of problem for the tenants that need it.

Per-tenant NAT gateways. One NAT gateway per tenant, with each tenant’s subnets routing to their own, gives attribution at the egress point and separates one tenant’s egress reputation from another’s. It costs an hourly charge per gateway and it is the fix for the attribution finding.

VPC Flow Logs with the right fields. Flow logs can include the interface, subnet, VPC and the AWS account of the ENI owner. Including the account field is what makes attribution possible in a shared VPC without a correlation step, and it is a configuration choice teams frequently miss.

VPC Lattice. Service-level authorisation between workloads, with IAM auth policies deciding whether one service may call another. Where the sharing requirement is really “these teams call each other’s APIs”, this expresses it at the layer where the requirement lives rather than at the packet layer.

Evaluation

Side by side

Control	Owned by	Tenant can weaken	Prevents accidents	Resists insider	Attribution
Security groups	Participant	✓ their own	✓	✗	Via ENI owner
Network ACLs	VPC owner	✗	✓	✓	✗
Route tables	VPC owner	✗	Egress only	Egress only	Via per-tenant NAT
SCPs on rule creation	Organisation	✗	✓	✓	n/a
Separate VPC + PrivateLink	Tenant	✗	✓	✓ strongly	✓
Per-tenant NAT	VPC owner	✗	n/a	n/a	✓
VPC Lattice auth policies	Service owner	Partly	✓	✓ at layer 7	✓ per request

The column that decides the architecture is “resists insider”. Security groups do not, because the tenant owns them, which means a legal separation requirement cannot rest on them. NACLs and SCPs do, and they are blunt. A separate VPC does, completely, which is why the two protected-data tenants belong somewhere else.

The solution

Move the two protected tenants out, use owner-controlled NACLs as the backstop for the rest, constrain what tenants can configure with SCPs, and give every tenant its own NAT gateway.

Start by questioning the arrangement for the tenants that need real separation. The shared VPC exists to conserve addresses, and the two protected-data workloads are not large. Giving each its own small VPC costs a modest allocation and removes the requirement from the network layer entirely: different VPCs share no routing domain, so no rule has to be written and no rule can be got wrong. Where those tenants need a service from the shared estate, a PrivateLink endpoint gives them that one service. This is the cheapest strong answer available and it is frequently rejected out of habit rather than analysis.

For the twelve remaining tenants, the goal is preventing accidents and constraining configuration. Add NACLs at the subnet level in the owner account expressing the coarse rules: which tenant blocks may reach which. They are stateless, so the ephemeral port ranges have to be right, and they are owner-controlled, so a participant cannot undo them. Keep them coarse; a NACL is a backstop against a badly written security group, not a substitute for one.

Constrain what tenants can create with SCPs. Deny the creation of security group rules with a source of 0.0.0.0/0 on anything except the ports the platform explicitly allows, and deny modification of the shared subnets and route tables outright, which the shared-VPC model mostly does already. This moves the control from “tenants write good rules” to “tenants cannot write the worst rules”.

Give each tenant its own NAT gateway with route tables directing their subnets to it. This is the attribution fix: an investigation reading the NAT gateway’s flow logs knows whose traffic it is without correlating against ENI ownership. It also stops one tenant’s outbound behaviour affecting another’s, which matters the first time a vendor rate-limits by source address.

Turn on flow logs with the ENI owner account field included, in the owner account, so attribution works inside the VPC as well as at the egress. This is a configuration detail with a large effect on how an investigation goes.

Then reduce the security group sprawl. Encourage groups that reference other groups rather than CIDR ranges, so a rule says “the payments API tier” rather than “10.4.0.0/16”, and the rule stays correct when addresses change. This does not resist an insider, and it removes most of the accidental exposure.

Why not rely on security groups alone. They are the control the tenants own, which makes them the wrong place to put a requirement that other tenants cannot access protected data. A tenant with a legal obligation cannot meet it with a rule another tenant could have written differently.

Why not put a firewall inside the VPC. Traffic between subnets in a VPC follows the local route, and a route table cannot send it elsewhere. Inspecting intra-VPC traffic means Traffic Mirroring or an agent on the instances, not an inline appliance, which is a common design mistake worth stating plainly.

Worked example

The two protected tenants move over six weeks. Each gets a /24 rather than the /16 the objection assumed, which turns the address-space argument from a blocker into a rounding error. Each takes two PrivateLink endpoints for the shared services it needs. Nothing else changes for them, and the legal requirement is now met by the absence of a route rather than by the presence of a rule.

The NACL work produces the expected stateless-rules confusion. The first version blocks return traffic because the ephemeral range was omitted on the outbound side, which is caught in a test subnet before it reaches anyone. The final rule set is nine entries per NACL and deliberately coarse.

The SCP denying wide security group sources finds four existing violations on deployment, all of them in non-production, all of them created by copying a rule from a tutorial. The teams fix them in a day, and the interesting part is that the SCP prevents the next one silently.

Per-tenant NAT gateways add fourteen hourly charges and immediately pay part of it back in reduced cross-zone transfer, because several tenants had been routing to a gateway in another Availability Zone.

The attribution improvement is proven three months later, when a vendor reports abusive traffic from one of the agency’s addresses. Under the old arrangement that would have been a day of correlating flow logs against ENI ownership. With per-tenant NAT it is one lookup, and the tenant is identified in about four minutes.

What’s worth remembering

1. A VPC is one routing domain and the local route cannot be removed, so subnets are not trust boundaries and no route table can isolate two subnets in the same VPC from each other.
2. In a shared VPC the owner account controls route tables, NACLs and the VPC; participants control their own resources and security groups. That split decides which controls a tenant can weaken.
3. A requirement that other tenants cannot access data cannot rest on security groups, because the tenants own them; owner-controlled NACLs, SCPs, or a separate VPC are the controls that resist the tenant.
4. Separate VPCs with PrivateLink is the strong isolation answer and usually costs far fewer addresses than assumed, so re-examine the address-space argument for the tenants that need real separation.
5. Isolation without attribution is incomplete: a shared NAT gateway makes an investigation a correlation exercise, and one gateway per tenant turns it into a lookup.
6. Intra-VPC traffic cannot be routed through an inline appliance, so inspecting it means Traffic Mirroring or host agents rather than a firewall in the path.