Few changes that I wish to suggest..
calshankar opened this issue · comments
Awesome project for learning & developing skills in Terraform. I would like to suggest following changes & would like to create pull request which i h'v tested but would like know whether it makes sense for review.
- custom LT for managed group. Many time users would like to pass custom configuration. I'm passing some of the recommened once https://blog.codeship.com/running-1000-containers-in-docker-swarm & below code block as part of node initialization. Besides the point, users can use what suits them.. Excuse me for long block
if ! grep -q imageGCHighThresholdPercent /etc/kubernetes/kubelet/kubelet-config.json;
then
sed -i '/"apiVersion*/a \ \ "imageGCHighThresholdPercent": 70,' /etc/kubernetes/kubelet/kubelet-config.json
fi
# Inject imageGCLowThresholdPercent value unless it has already been set.
if ! grep -q imageGCLowThresholdPercent /etc/kubernetes/kubelet/kubelet-config.json;
then
sed -i '/"imageGCHigh*/a \ \ "imageGCLowThresholdPercent": 50,' /etc/kubernetes/kubelet/kubelet-config.json
fi
## Initializing kubelet based on spot/ondemand
instance_id=$(curl -s http://169.254.169.254/latest/meta-data/instance-id)
instance_type=$(aws ec2 describe-instances --instance-ids $instance_id --query 'Reservations[0].Instances[0].InstanceLifecycle' --output text)
export AWS_REGION=$(curl -s http://169.254.169.254/latest/dynamic/instance-identity/document | jq -r .region)
if [ "$instance_type" == "spot" ]; then
/etc/eks/bootstrap.sh '${CLUSTER_NAME}' --b64-cluster-ca '${B64_CLUSTER_CA}' --apiserver-endpoint '${API_SERVER_URL}' --kubelet-extra-arg "--system-reserved cpu=250m,memory=0.2Gi,ephemeral-storage=1Gi --kube-reserved cpu=250m,memory=1Gi,ephemeral-storage=1Gi --eviction-hard memory.available<0.2Gi,nodefs.available<10% --allowed-unsafe-sysctls net.core.somaxconn,net.ipv4.tcp_tw_reuse --event-qps=0 --read-only-port=0"
else
/etc/eks/bootstrap.sh '${CLUSTER_NAME}' --b64-cluster-ca '${B64_CLUSTER_CA}' --apiserver-endpoint '${API_SERVER_URL}' --kubelet-extra-arg "--system-reserved cpu=250m,memory=0.2Gi,ephemeral-storage=1Gi --kube-reserved cpu=250m,memory=1Gi,ephemeral-storage=1Gi --eviction-hard memory.available<0.2Gi,nodefs.available<10% --allowed-unsafe-sysctls net.core.somaxconn,net.ipv4.tcp_tw_reuse --event-qps=0 --read-only-port=0"
fi
-
Local null provisioner to setup kubectl access by default for the role being used. assume kubectl is already setup
-
Support for automatically adding spot node-group role added to auth config map for node to show up automatically show up in the cluster. Currently it has to be done manually..
-
The cluster is opened up for public access which is not good security posture for the cluster. I made the change to support access from local runner/instance public IP to limit the access
Hi @calshankar, points 1 and 4 are definitely something nice to add.
Points 2 and 3 were something I wanted to added, however I think it is much cleaner when we don't add any local provisioners for kubectl. With regards to configmap, it will need to be added to terraform-k8s as I don't want to mix aws and kubernetes providers. I have ran into mega issues previously.