Configure the non-server parts of an AWS datacenter without using CloudFormation. Multiple VPCs in multiple regions can be configured with one pillar.
Requires: Saltstack 2016.11 ( 2015.8.2 if not using NAT Gateway ) and Boto3
Note
See the full Salt Formulas installation and usage instructions.
Create VPC objects, including:
- VPC
- Internet Gateway
- NAT Gateway
- Subnets
- Routing Tables
Create ec2 components including:
- Key pairs
Create and update Security Groups rules. Servers must be added to the Security Groups as part of the salt-cloud creation proces.
The salt boto states used in this formula have some limitations that do not allow for the one pass full creation of a VPC datacenter. The current issues are:
- NAT Gateways can not be added to a routing table by name. ( Found the undocumented nat_gateway_subnet_name option that would resolve this issue, but it is not working per Salt Issue 38791. )
- Inter-Region VPN connections can not be added to routing table
NAT Gateway Workaround
The workaround for the nat gateway is to do an initial salt state run to create the gateways, then add the NAT Gateway's nat_gateway_id to the routing tables in the pillar and re-run the state. In the sample pillar the default Gateways are commented out to indicate where the nat_gateway_id should go.
Inter-Region VPN
AWS currently does not offer a service for VPC peering between regions. This means the recommended solution is to create a VPN instance and then use an ipsec VPN tunnel between regions. Unfortunately, when using VPN instances you can not use the Route Propagation functionality, so those routes have to be manually added to all routing tables for a VPC. VPN routing is shown commented out in the pillar example. Like above, run the states once to create all of the objects. Then when the VPN tunnel is created/active, add the IPs to the pillar and rerun the states.
All configuration is done through the AWS pillar. The hierarchy of this pillar is:
aws:
region:
us-east-2:
key_pairs:
profile:
vpc:
myvpc:
vpc:
internet_gateway:
subnets:
routing_tables:
routing_global_routes:
security_groups:In this hierarchy, the 3rd level ( us-east-2 ) is a region name and the 5th level ( myvpc ) is a vpc name. These are the names that will be used for the region and the name of the VPC. All items besides internet_gateway can have multiple values.
The below examples and the sample pillar uses single quotes in many places to ensure data is interepreted correctly. Not using quotes per the examples is done at your own risk.
Key pairs are included under at the region level since they are not generally VPC specific. Key pair format is a key pair with the name and RSA publi key.
key_pairs:
mykey: 'ssh-rsa XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX admin@mycompany.com'This formula uses an AWS profile for all states instead of the individual fields. The key and keyid should be gpg encrypted using the Saltstack gpg renderer. Example below shows it in unencrypted format.
profile:
region: us-east-2
keyid: ASDFASDFASDFASDFASDF
key: AB12Cd3Efg45hIjk67lMNop8q9RST0uvwXyzVPC contains vpcs for a given region. Each vpc will have data for all VPC specific states, even if they are not in the vpc.sls. The vpc pillar name is the name that will be used for the VPC in AWS. The only data directly under the vpc name is the CIDR block for the VPC. This Formula is designed using a class B network for the VPC and class C for all subnets.
vpc:
myvpc:
cidr_prefix: '10.10'the VPC subsection contains the data needed to create the VPC. The names on the left are the configuration item names from the boto_vpc.present states. The vpc pillar name should always match the name in the vpc section beneath. The cidr_block should start with the same two octets as the cidr_prefix above.
vpc:
myvpc:
vpc:
name: myvpc
cidr_block: 10.10.0.0/16
instance_tennancy: default
dns_support: 'true'
dns_hostnames: 'true'An internet gateway is needed for most use cases.
vpc:
myvpc:
internet_gateway:
name: internet_gatewaySubnets are named by their subnet ID ( assumes we are using class C subnets). The subnet ID will be appended to the cidr_prefix above to create the CIDR or the subnet. Every subnet has to at least have a subnet name and availability zone. if nat_gateway is specified, then a NAT Gateway will be created in that subnet. Subnet associations are done in the Routing Table section below.
vpc:
myvpc:
subnets:
1:
name: subWebA
az: a
nat_gateway: true
11:
name: appwebA
az: aThe above example would create two subnets:
- subWebA with CIDR 10.10.1.0/24
- subAppA with CIDR 10.10.1.0/24
Both are in Availability Zone A and a NAT Gateway would be created in subWebA.
Routing tables will create the tables, add routes, and assign subnets to routing tables. The below example include the interface_id of a already created NAT Gateway.
vpc:
myvpc:
routig_tables:
publicA:
routes:
default:
destination_cidr_block: 0.0.0.0/0
internet_gateway_name: internet_gateway
subnet_names:
- subWebA
privateA:
routes:
default:
destination_cidr_block: 0.0.0.0/0
nat_gateway_id: 'nat-0abcdef123546ghi'
subnet_names:
- subAppARoutes that will be added to all routing tables. Use this for adding vpn routes.
vpc:
myvpc:
routing_global_routes:
vpnPROD:
destination_cidr_block: '10.10.0.0/16'
instance_id: 'i-xxxxxxxxxxxxxxx'Create security groups and rules. Usage notes:
- If a single port is being specified, the from_port and to port can be replace with just port.
- source_group_name and cidr_ip can be either a single item or a list.
- Use port: -1 to specify all ports
- A rules pillar name is for information purposes only and is not used in the actual rule creation.s
vpc:
myvpc:
security_groups:
sgApp-myvpc:
description: SG for all App servers
rules:
http:
ip_protocol: tcp
port: 80
source_group_name:
- sgWeb-myvpc
- sgApp-myvpc
rules_egress:
all:
ip_protocol: all
port: -1
cidr_ip: '0.0.0.0/0'
sgSalt-myvpc:
description: SG for all Salt servers
rules:
salt-master:
ip_protocol: tcp
from_port: 4505
to_port: 4506
cidr_ip: '10.10.0.0/16'
salt-api:
ip_protocol: tcp
port: 443
cidr_ip:
- '10.10.0.0/16'
- '10.20.0.0/16'