My Experience with AWS VPC Private Subnets: Implementation and Lessons Learned 🚀💻

image source: https://docs.aws.amazon.com/images/vpc/latest/userguide/images/vpc-example-private-subnets.png

Overview of the project:

In my recent project, I explored the implementation of AWS VPC Private Subnets. This setup included both public and private subnets distributed across two availability zones. Each public subnet was equipped with a NAT gateway and a Load Balancer node. The servers, which are EC2 Instances, operated within the private subnets. These instances were dynamically managed through an Auto Scaling Group, receiving traffic from the Load Balancer. Additionally, they accessed the internet via the NAT gateway. This experience provided valuable insights and lessons that I am eager to share.

The topics I learned from this project include:

1. What is a NAT gateway and how it works ?

A NAT gateway (Network Address Translation) is a network service that enables multiple servers, such as EC2 Instances on a private network (private subnets), to use a single public IP address for accessing the internet or other external networks.

How it works:

• Servers on the private subnets send requests to the NAT gateway.

• The NAT gateway then translates the private IP address of these servers to the public address of the NAT gateway.

• The NAT gateway forwards the request to the external network or the internet.

• When the external network or internet responds, the NAT gateway translates the response back to the private IP address of the original device.

So the NAT gateway provides security because the servers on the private subnet are not exposed to the internet.

2. What is an Auto Scaling Group ?

An Auto Scaling Group (ASG) is a feature in AWS, that allows you to automatically scale your EC2 Instances based on the demand. For Example: you have an e-commerce website that sells products online. You except a lot of traffic on your website during holidays like Diwali. Your website might become slow or unresponsive if too many people visit the website at same time. So you create an Auto Scaling Group with the following settings:

• Minimum Instances will be 2 (In this case you have at least 2 servers running).

• Maximum Instances will be 10 (In this case you can scale up to 10 servers if needed).

• Desired Instances will be 5 (In this case you start with 5 servers).

3. What is a Load Balancer?

A Load Balancer is a technology that distributes incoming network traffic across multiple servers to improve the responsiveness, reliability, and scalability of an application.

4. What is a Target Group?

A target group is a list of servers. When someone visits your website, the load balancer looks at the list of servers in the target group and chooses one to send the visitors to. Here, the load balancer ensures that:

• Each server in the target group is working correctly.

• Visitors are spread evenly across all the servers.

• If one of the servers gets busy, the load balancer can send visitors to another server.

This ensures that your website stays available and responsive, even if you get a lot of visitors at the same time.

5. What is a bastion host ?

A Bastion Host, also known as a "jump server," is a secure intermediate server that acts as a gateway to access other servers or resources within a private subnet.

Project Implementation:

1. Create the VPC

To create the VPC, follow these steps:

• Open the Amazon VPC Console.

• On the dashboard, choose Create VPC.

• Select VPC and more.

• Configure the VPC:

  • For Name tag auto-generation, enter a name for the VPC.

  • For IPv4 CIDR block, keep the default suggestion.

• Configure the Subnets:

  • For Number of Availability Zones, choose 2.

  • For Number of public subnets, choose 2.

  • For Number of private subnets, choose 2.

• For NAT gateways, choose 1 per AZ.

• For VPC endpoints, select None if you don’t require any S3 Bucket.

• For DNS options, clear Enable DNS hostnames.

2. Create an Auto Scaling Group

To create an Auto Scaling Group, you first need to create a Launch Template. Follow these steps:

1. Create a Launch Template:

   • Go to the AWS Console Dashboard and select the EC2 service.

   • On the left side, at the bottom, find the "Launch Templates" option. Click on it and then select "Create Launch Templates."

   • Provide a Name and Description for the Launch Template.

   • Choose your preferred AMI (Amazon Linux, Ubuntu); In my case, I selected the Ubuntu AMI.

   • Select the Instance type.

   • Create a Key Value Pair.

   • Configure the Network Settings:

• Select "Create Security Group."

• Provide a Name and Description for the Security Group.

• Choose the VPC you have created.

• Edit the Inbound Security Group Rules as needed.

• Now, select "Create Launch Template."

2. Create an Auto Scaling Group:

   • Go to the EC2 Instance section, and at the bottom left, click on "Auto Scaling Group."

   • Select "Create Auto Scaling Group."

   • Provide a Name for the Auto Scaling Group.

   • Select the Template you just created.

   • Click on "Next."

   • Configure the Network Settings:

• Select the VPC you have created.

• In the Availability Zones and Subnets, select both the private subnets since the application will be deployed there.

• Click on "Next."

• Load Balancer and Group Size:

• Select "No Load Balancer" and click on "Next."

• Set the Group Size:

  • Desired Capacity: Choose 2.

  • Minimum Capacity: Choose 1.

  • Maximum Capacity: Choose 4.

  • Select "No Scaling Policies" and click on "Next."

• Finalize:

• Add Notifications and Tags (both are optional).

• Review all the resources and click on "Create Auto Scaling Group."

After completing these steps, you will have 2 EC2 Instances created in your Instances section. Both the instances which are created using Auto Scaling Group has no public IP address.

Now to deploy our application inside this servers which are located inside the private subnets we required a bastion host or a jump server which will act as a gateway to access the servers (EC2 Instances) within a private subnet.

3. To create a Bastion Host:

   • Go to the AWS Console Dashboard and select the EC2 service.

   • Choose the same AMI and Instance type as before.

   • Use the same key pair you created earlier.

   • Select the VPC you have already set up

   • Place this Bastion host in the public Subnet.

   • Enable Auto-Assign Public IP.

   • Click on Launch Instance.

   • To log into the bastion server if using MobaXterm :

     1. Open MobaXterm and start a new session.

     2. Choose the "SSH" option.

     3. Enter the public IP address of your bastion server in the "Remote host" field.

     4. In the "Specify username" field, enter the username (e.g., ubuntu).

     5. Go to the "Advanced SSH settings" tab.

     6. In the "Use private key" field, browse and select the .pem file you copied to your remote directory.

     7. Click "OK" to start the SSH session.

This will log you into the bastion server, allowing you to access resources within the private subnet.

Next, copy the .pem file from your local machine to the remote bastion server to access servers within private subnets. If using MobaXterm, execute this commands in the MobaXterm local terminal:

• chmod 400 /home/mobaxterm/Desktop/name.pem to set the file permissions of "name.pem" to read-only for the owner, enhancing security.

• scp -i /home/mobaxterm/Desktop/name.pem /home/mobaxterm/Desktop/name.pem ubuntu@public-ip-address:/home/ubuntu to copy the .pem file to the bastion server.

• Using the above scp command, you will have a copy of .pem file into the bastion server.

To access your application within the private subnet, use the below SSH command inside the bastion server:

• ssh -i name.pem ubuntu@private-ip-address

Use the private IP address of an instance created using the Auto Scaling Group. This command allows you to log into the instance within the private subnet. I created a Portfolio website using HTML code inside a Vim file and ran a simple Python server inside this instance using:

• python3 -m http.server 8000

This command starts a simple HTTP server using the http.server module in Python 3.

To receive requests from the outside world into your application, create a Load Balancer and Target Group:

4. Create the Target Group:

• Go to the EC2 service, and on the left bottom side, find the Target Group option. Click on it and create a Load Balancer. Choose the Application Load Balancer option.

• Select “Instances” as the target type.

• Provide a Name for the target Group and specify the port number (e.g., 8000).

• Click Next.

• Select the instances in the private subnets and click on include as pending.

• Create the Target Group.

5. Create the Load Balancer:

• Go to the EC2 service, and on the left bottom side, find the Load Balancer option. Click on it and create a Load Balancer. Choose the Application Load Balancer option.

• Provide a Name and ensure it is Internet-Facing.

• Choose IPV4 for the Load Balancer IP address type.

• Select the same VPC you created earlier.

• Tick both availability zones and ensure the subnets are Public.

• Select the same security group created during the launch template.

• Select the Target Group that you created in the Listeners and Routings section of the Load Balancer.

• Edit the security inbound rule of the Load Balancer to add port 8000.

• Create the Load Balancer.

To access the application through the internet, enter http://dns-name:8000 in your browser. You should now be able to view the application online.

This is the complete implementation of this project. You can also refer to this video where I learned all these topics and implemented this project. The link to the video is https://youtu.be/FZPTL_kNvXc?si=8w17H9EY-oZAu0Qa.