AWS NAT Gateway does not let EC2 connect to Internet

Question

I have a pre-existing VPC with a subnet CIDR of 172.31.80.0/20.

There is an Internet Gateway that has this VPC attached to it.

I created the following

• NAT Gateway, it has a public IP and a private IP (172.31.90.191) attached to the same VPC

• Routing Table with the same VPC and three routes:

    1) 172.31.0.0/16 target local
  
    2) 0.0.0.0/0 target the NAT Gateway
  
    3) 172.32.0.0/16 target of PCX for my other VPC were my EC2 VPN is
  

The NAT is associated to the subnet and the subnet is associated to the routing table.

The ACL on the VPC, Routing Table & subnet all have allow all traffic all protocols all ports 0.0.0.0/0 both in and out.

The Amazon Linux 2 EC2 has security groups to allow all traffic all protocols all ports 0.0.0.0/0 both in and out.

When it boots cloud-init assignes it a default route of 172.31.80.1. It cannot ping 8.8.8.8 and cannot curl google.com

I truned on FlowLogs to capture all logs, and for the ENI of the EC2 the logs only show ACCEPT OK for packets going back and forth between the VPN and the EC2.

I even see ACCEPT OK for a 'dig google.com @8.8.8.8'

eni-???? 172.31.90.208 8.8.8.8 50974 53 17 6 402 1600966519 1600966524 ACCEPT OK

But I don't see a reply comming back.

Looking at the details for everything shows statuses of Avaliable and Active, all green.

What is wrong with my gateway?

Is cloud-init giving me the wrong gateway IP?

I understand that the VPC must be behind a NAT on public subnet.

As I understand it, the only difference between a public and private subnets is that a public subnet is associated to a routing table that routes 0.0.0.0/0 to an Internet exposed destination.

My routing table has 0.0.0.0/0 with a destination of the NAT and it has a public IP. Is something more needed?

Answer 1

The NAT needs to have access to the internet (it is just another resource in your network after all).

To do this ensure you place it in a public subnet (has a route table which uses an internet gateway for its 0.0.0.0/0 route).

When people use a NAT it will stay in a public subnet, with the resources that would use the NAT being located in private subnets with route tables that use the NAT as a destination.