How To Create Subnet In Aws

Connecting to an ec2 instance in a private subnet on AWS

This article is a 'how to guide' on connecting to instances in a private subnet on AWS. It includes an example of the required infrastructure in Terraform and making changes to your SSH configuration file.

If you'd like to skip ahead, all the code discussed in this post is available on my GitHub here.

Amazon Virtual Private Cloud (Amazon VPC) enables you to launch AWS resources int o a virtual network that you've defined. A subnet is a range of IP addresses within the VPC. Subnets can be either public with a gateway to the internet or private. Instances launched in a public subnet can send outbound traffic to the internet while instances launched in the private subnet can only do so via a network address translation (NAT) gateway in a public subnet. Naturally private subnets are more secure, as the management ports aren't exposed to the internet. Typically in a modular web application, the front end web server will reside within the public subnet while the backend database is in the private subnet.

There are many reasons why connecting to instances in a private subnet is necessary:

The backend database for an application resides within a private subnet and an engineer needs access to perform ad-hoc analysis.
The private subnet is whitelisted against another third-party service and it's a requirement to interact with that service.

Connecting to a private subnet

Instances within the same VPC can connect to one another via their private IP addresses, as such it is possible to connect to an instance in a private subnet from an instance in a public subnet; otherwise known as a bastion host.

Amazon instances use SSH keys for authentication. As such connecting to the private instance will require a private key on the bastion host; in the same way connecting to the public instance requires a private key on your host machine, however this is extremely bad practise. Never expose your private keys to a bastion host!

An alternative solution is to use SSH agent forwarding, which allows a user to connect from the bastion to another instance without storing the private key on the bastion. An SSH agent is a program that keeps track of user's identity keys and their pass phrases and can be configured with the following commands:

          # Generate SSH keys
ssh-keygen -k mykeypair          # Add the keys to your keychain
ssh-add -K mykeypair

Once the keys have been generated and added to the keychain, it is possible to connect to the bastion instance with SSH using the -A option. This enables forwarding and lets the local agent respond to the public-key challenge when connecting to instances from your bastion.

          # Connect to the bastion host:
ssh -A <bastion-ip-address>

the next step is deploying the required infrastructure using Terraform.

Infrastructure

The infrastructure below has been deployed using Terraform; an open-source infrastructure as code software (and the best thing since sliced bread!).

Diagram created using: https://creately.com

VPC: An Amazon VPC (virtual private cloud) is an isolated section of the AWS cloud where you can provision your infrastructure. In all likelihood your organisation will already have a VPC, if this is the case you can skip this step.

Subnets: Subnets are essentially subsets of available addresses in your VPC and add an extra layer of control and security to resources in your environment. As above your organisation will probably already have subnets set up, but if not, please see below.

The key differentiator between a private and public subnet is the map_public_ip_on_launch flag, if this is True, instances launched in this subnet will have a public IP address and be accessible via the internet gateway.

Secondary CIDR: If all of your organisation's IP addresses in its VPC are occupied by private subnets, a way around this is to create a secondary CIDR block and launch a public subnet in it.

Internet Gateway: For a subnet to be accessible to the internet an AWS internet gateway is required. An internet gateway allows internet traffic to and from your VPC.

Route table: A Route table specifies which external IP address are contactable from a subnet or internet gateway.

Nat Gateway: A Nat Gateway enables instances in private subnets to connect to the internet. The Nat gateway must be deployed in the public subnet with an Elastic IP. Once the resource is created, a route table associated with the the private subnet needs to point internet-bound traffic to the NAT gateway.

Security Groups: A security group acts as a virtual firewall for your instance to control incoming and outgoing traffic. The security group below enables all traffic over port 22 (SSH). Both instances in the private and public subnet require this security group.

Ec2 Instances and Keys: After all the necessary infrastructure has been defined, we can set up our Ec2 instances. The instances require an AWS key-pair to authenticate access which is created below using the aws_key_pair resource and existing ssh key created earlier.

Now that the infrastructure is complete the next step is to deploy. This can be achieved with the following Terraform commands in the terraform directory:

          terraform init          terraform apply

If the deployment has been successful, you'll be able to see two new EC-2 instances in your AWS console.

The SSH config file

The SSH config file is a great resource for storing all your configuration for the remote machines you connect to. It is located in your home directory here: .ssh/config. The config file isn't automatically created, so if it doesn't exist you will have to create it.

          Host bastion-instance
            HostName <Bastion Public IP>
            User ubuntu          Host private-instance
            HostName <Private IP>
            User ubuntu
            ProxyCommand ssh -q -W %h:%p bastion-instance

The ProxyCommand in the private-instance tells SSH to make a connection to the bastion-instance and then establish a TCP forwarding to the private-instance.

Finally, you can connect to the private instance over SSH using the command below:

          ssh private-instance

And Voila, you're in!

I hope you enjoyed this post; if you have any questions or suggestions, or even ideas for future posts, let me know in the comments section and I'll do my best to get back to you.

Please checkout my other posts:

https://towardsdatascience.com/apache-airflow-automating-the-collection-of-daily-email-attachments-213bc7128d3a
https://towardsdatascience.com/selenium-on-airflow-automate-a-daily-online-task-60afc05afaae