This is derived from Learn Terraform Provisioner - EKS Cluster and its companion tutorial. Significant modifications were made to allow configuration dependencies to be generated by Terraform directly.
Objectives:
- Provision full-featured Kubernetes clusters on AWS with sensible defaults
- Minimize infrastructure costs
Notable defaults:
- Fully encrypted disks & secrets in cluster
- Spot instances and autoscaling
Notable options for configuration:
- Autoscaling can be disabled
- NAT gateway usage can be disabled
Tested upgrade path from Kubernetes 1.20 to 1.21. Upgrades are slow, mainly due to unobservable control plane processes in EKS.
Configure your cluster details in terraform.tfvars.json
.
Create the cluster:
bash -x up.sh
When you reconfigure your cluster, either by modifying Terraform files or the variables:
bash -x up.sh
NB: because the autoscaler will modify the desired size of each autoscaling group, Terraform will report changes having been made outside of Terraform's operations. This is normal, and shouldn't impact application operations.
Destroy the cluster:
bash -x down.sh
- A nice CLI wrapper, perhaps
- A pre-generated admin service account for attaching other management apps
- Smarter automation of security groups
- Cleaner configuration of node groups (right now it seems really disjoint to me)
- Validate the behavior of running applications (e.g. a database) during upgrade