A VPC (Virtual Private Cloud) is a Software Defined Network (SDN). It is an AWS service that provides AWS customers with means to provision a logically separated section of AWS Cloud. By creating a VPC you are creating your own network within AWS.
You may have to design multiple VPCs for your projects and environments. Your VPCs are the fundamentals of your cloud infrastructure. If you create a solid, future-proof design now, you will avoid expensive and time-consuming adjustments that may disturb your business operations when your business grows. How you design your VPC will have a direct impact on flexibility, scalability, security and fault-tolerance of your cloud infrastructure.
In this article I will briefly present VPC design considerations for an n-tier web application:
Choose your Virtual Private Cloud size and network CIDR wisely
The first thing to consider is the size of your VPC. The allowed CIDR block size is between /28 netmask (16 IP addresses) and /16 netmask (65536 IP addresses). It is important to realise that you cannot change the size of your VPC once you have created it. If you run out of IP addresses, you will have to delete your VPC and create a larger one. That, of course, means that any EC2 instances and other resources will have to be terminated and re-launched. Therefore, to achieve as much flexibility as possible, I allocate the maximum possible number of addresses to the VPC.
For the sake of this example I will use 172.20.0.0/16 network that gives me 65536 addresses and an address range of 172.20.0.0 – 172.20.255.255. It should be noted that any potential conflicts with other networks that may be connected via virtual private gateway to your VPC should be considered. CIDR should be chosen to avoid such conflicts. You do not want to use 10.0.0.0/16 network if your local datacentre uses 10.0.26.0/24 for example.
/16 network with its 65536 addresses may appear too big for you especially if you have multiple networks within your organisation occupying various private IP address spaces. If /16 seems to you like a waste of precious space and you think that you will never need the enormous number of IP addresses in your VPC, consider smaller networks such us /18 or /20.
Geographical locations matters
I assume that the majority of my visitors will be coming from the UK. I will therefore build my infrastructure in the EU west 1 region (Ireland) due to its closest proximity to the UK. There are currently three availability zones (different physical locations) in the EU west 1 region. To achieve the highest possible fault tolerance in this region, I will use all three availability zones. I divide the address space evenly across them but also leave a spare block of IP addresses to keep the set up flexible:
172.20.0.0/18 – Availability Zone a (172.20.0.0 – 172.20.63.255)
172.20.64.0/18 — Availability Zone b (172.20.64.0 – 172.20.127.255)
172.20.128.0/18 — Availability Zone c (172.20.128.0 – 172.20.191.255)
172.20.192.0/18 — Spare (172.20.192.0 – 172.20.255.255)
Each availability zone has now 16382 IP addresses and there are 16382 spare IP addresses. I keep the spare addresses just in case. I may allocate them in the future should it be required.
Subnets layout is important
I place AWS resources in subnets depending on how they need to be accessed. Generally speaking, it is a good idea to create a subnet if there is a need for different hosts to route differently or if it is advantageous to use network ACL on a subnet level. Subnets cannot span across availability zones so I will have to create a subnet in each of the three availability zones per each tier of my application. Taking all that into consideration I create three subnets in each availability zone:
- Public subnet – only public facing resources that requires full internet access should be deployed into this subnet. In the case of my application only elastic load balancers will be placed in this subnet.
- Private subnet for application tier– EC2 instances hosting the web application will be placed in this subnet.
- Private subnet for data tier – any data stores such as RDS and ElasticCache will be placed in this subnet.
I will have 9 subnets in my VPC (3 tiers x 3 availability zones) I will allocate the same number of IP addresses with each subnet and I will still leave some spare IP addresses in each availability zone to keep things flexible. I will therefore use CIDR /20 subnets which will give me 4094 IP addresses in each subnet. Therefore, there will be 12282 IP addresses per tier (3 availability zones x 4095 IP addresses each).
My design will look like this:
172.20.0.0 / 16 – VPC – 65536 IPs (172.20.0.0 – 172.20.255.255)
172.20.0.0/18 – Availability Zone a – 16382 IPs (172.20.0.0 – 172.20.63.255)
172.20.0.0/20 – Public Subnet –4094 IPs (172.20.0.0 – 172.20.15.255)
172.20.16.0/20 – Private Subnet App Tier –4094 IPs (172.20.16.0 – 172.20.31.255)
172.20.32.0/20 – Private Subnet Data Tier – 4094 IPs (172.20.32.0 – 172.20.47.255)
172.20.48.0 – 172.16.63.255 – Spare – 4094 IPs
172.20.64.0/18 — Availability Zone b – 16382 IPs (172.20.64.0 – 172.20.127.255)
172.20.64.0/20 – Public Subnet –4094 IPs (172.20.64.0 – 172.20.79.255)
172.20.80.0/20 – Private Subnet App Tier – 4094 IPs (172.20.80.0 – 172.20.95.255)
172.20.96.0/20 – Private Subnet Data Tier – 4094 IPs (172.20.96.0 – 172.20.111.255)
172.20.112.0 – 172.20.127.255 – Spare – 4094 IPs
172.20.128.0/18 — Availability Zone c – 16382 IPs (172.20.128.0 – 172.20.191.255)
172.20.128.0/20 – Public Subnet –4094 IPs (172.20.128.0 – 172.20.143.255)
172.20.144.0/20 – Private Subnet App Tier – 4094 IPs (172.20.144.0 – 172.20.159.255)
172.20.160.0/20 – Private Subnet Data Tier – 4094 IPs (172.20.160.0 – 172.20.175.255)
172.20.176.0 – 172.20.191.255 – Spare – 4094 IPs
172.20.192.0/18 — Spare – 16382 IPs (172.20.192.0 – 172.20.255.255)
You may of course, get to a different conclusion when you consider your n-tier web application. For example:
- Larger App tier
You may conclude that it is more likely that your application tier will contain much more resources (EC2 instances) than any other tiers. In this case you may prefer having larger app tier subnets and smaller web tier and data tier subnets. You could therefore lie out your subnets the following way:
172.20.0.0 / 16 – VPC – 65536 IPs (172.20.0.0 – 172.20.255.255)
172.20.0.0/18 – Availability Zone a – 16382 IPs (172.20.0.0 – 172.20.63.255)
172.20.0.0/19 – Private Subnet App Tier –8190 IPs (172.20.0.0 – 172.20.31.255)
172.20.32.0/21 – Public Subnet Web Tier –2046 IPs (172.20.32.0 – 172.20.39.255)
172.20.40.0/21 – Private Subnet Data Tier – 2046 IPs (172.20.40.0 – 172.20.47.255)
172.20.48.0 – 172.20.63.255 – Spare – 4094 IPs
Of course you would replicate the same structure across all availability zones as in the previous design.
- One set of subnets for application and data tiers
You may conclude that regardless of how many application tiers you have, your VPC only needs one public tier (accessible from the internet) and one private tier (not accessible from the internet). You will launch your load balancers into your public set of subnets. Your application EC2 instances and databases will be in the same private set of subnets. There will not be a separate set of subnets for the application and the data.
This kind off flat private tier that contains mixed resources is absolutely fine if you do not have any particular reason to split your resources into separate subnets. It also has one big advantage. Namely, you won’t waste IP addresses that otherwise would have to be assigned to subnets but not actually used. Think about auto-scaling groups. You may have many EC2 instances but fewer RDS instances however you will never know the exact numbers. If you create two private tiers, you will simply have to guess the numbers like in the previous example where we created our application tier to be twice as large as our data tier. If you create one private tier, all the IP addresses can be allocated to any type of resource.
Your subnets could by laid out like this:
172.20.0.0 / 16 – VPC – 65536 IPs (172.20.0.0 – 172.20.255.255)
172.20.0.0/18 – Availability Zone a – 16382 IPs (172.20.0.0 – 172.20.63.255)
172.20.0.0/19 – Private Subnet App/Data Tier –8190 IPs (172.20.0.0 – 172.20.31.255)
172.20.32.0/20 – Public Subnet Web Tier –4096 IPs (172.20.32.0 – 172.20.47.255)
172.20.48.0 – 172.20.63.255 – Spare – 4094 IPs
Again, you would replicate the same structure across all availability zones.
Routing considerations
All instances in the VPC can communicate with each other by default. We do however have public and private subnets that require different routing tables.
Resources in the public subnet needs to be directly accessible from the internet. I therefore create an Internet Gateway in my VPC and a route table in which the Internet Gateway is the default route. I then associate the route table with all three public subnets. I only launch elastic load balancers into the subnets as these are the only resources, at least initially, that should be accessible from the internet. There is no need for any application EC2 instances to be in the public subnets (unless there is a good reason, for example bandwidth intensive processes that struggle behind NAT instances or gateways). They actually should not be there as normally they are only meant to accept traffic from load balancers only.
Although instances in the private subnets should not be directly reachable from the internet, they require outbound internet access. Normally, I would create NAT instances in each of the public subnets however by the end of the last year (December 2015) AWS released managed NAT gateways. Using a managed NAT gateway is simpler than creating your own NAT instances, therefore in most cases I would now go with this option. Unfortunately, unlike the Internet Gateway, NAT gateways do not have the same multi-zone capability. Therefore, I have to create a NAT gateway in each of the public subnets and create route tables with routes to the NAT gateways that I associate with private subnets.
There are other routing options that you may need to consider when designing a VPC for your n-tier web application. You may need a VPN connection between your office or your data centre and your VPC. In this case you have to create a customer gateway and virtual private gateway. Then, you create a route to your virtual private gateway in the subnets where you will launch instances that should communicate directly with your other networks.
You may also want to create and route traffic to VCP endpoints. They allow you to direct traffic from your VPC to S3 (and other AWS services in the future) directly without having to access the service over the internet.
In more advanced set-ups you may want to use VPC Peering (allowing communication between two VPCs) in which case you will also have to configure your routing tables to enable communication.
All these routing options should be kept in mind when you first design your VPCs. If you properly allocate IP addresses in the first place so that they do not overlap any other networks or other VPCs and so that you have enough room for growth in each of them, your business will have a strong fundamentals in the cloud.
I understand now how I should distribute the IP addresses between subnets but I am still not sure how large my VPC should be. You don’t give a definite answer of how to determine the CIDR of a VPC.
I cannot possibly give you a definite answer. The CIDR of your VPC will depend on some aspects of your projects as same as the network architecture in your organisation.
Good article. I just wonder why I would ever need a /16 network with 65536 IP addresses. Seems like a massive waste of space. Any example of when I would need a large VPC?
One application for a large VPC would be ad-hoc dev / test environments. Imagine that you have a large team of developers working on a project. Each of them would commit code to his/her feature branch in the code repository. They may want to be able to run and test the code in a production-like environment before merging and releasing their code.
You could design a system that would automatically spin off dev / test environments very similar to your production environment. Each of the environments would be launched in your VPC and would run code from a corresponding branch in your repository. The environment will be up until the code from the corresponding branch is tested and merged into your release branch.
Note that each environment may contain multiple resources and require multiple IP addresses. A large VPC would give you the flexibility to run many ad-hoc environments so that when your team grows, you can accommodate more environments without worrying about running out of space.
Ok, this makes sense but if all test/dev environments run in one VPC how would you isolate them from each other?
It depends what sort of isolation you require. I could imagine that if you work, for example, on a three tier web application, each of your test environments could just be an ELB, a couple of EC2 instances and an RDS instance. Each of the environments would be logically isolated by an auto-scaling group. RDS could be shared across your environments to cut costs. You would use Route 53 to assign urls. It is quite simple but very reliable and robust if you use terraform or cloud formation template to define and control all resources.
Ad-hoc dev environments on AWS…. It sounds like a good topic for an article 🙂 If you have experience building that, I would love to read about it. Testing multiple branches of code is a nightmare here where I work…
I second that!
Regarding NAT Gateways and multi-AZ, Gateway can support multiple AZs but that is not the recommended practice: “Create a NAT gateway in each Availability Zone to ensure zone-independent architecture.”
Yes, I agree. I can create a NAT gateway in one AZ and route the traffic from all my subnets in multiple AZs through that NAT gateway.
What I meant was that it would be better if a NAT gateway was designed similarly to an internet gateway. One internet gateway is enough to ensure a zone-independent architecture.