marcincuber / eks

AWS EKS - kubernetes project and terraform module

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Few changes that I wish to suggest..

calshankar opened this issue · comments

Awesome project for learning & developing skills in Terraform. I would like to suggest following changes & would like to create pull request which i h'v tested but would like know whether it makes sense for review.

  1. custom LT for managed group. Many time users would like to pass custom configuration. I'm passing some of the recommened once https://blog.codeship.com/running-1000-containers-in-docker-swarm & below code block as part of node initialization. Besides the point, users can use what suits them.. Excuse me for long block
if ! grep -q imageGCHighThresholdPercent /etc/kubernetes/kubelet/kubelet-config.json;
then
    sed -i '/"apiVersion*/a \ \ "imageGCHighThresholdPercent": 70,' /etc/kubernetes/kubelet/kubelet-config.json
fi

# Inject imageGCLowThresholdPercent value unless it has already been set.
if ! grep -q imageGCLowThresholdPercent /etc/kubernetes/kubelet/kubelet-config.json;
then
    sed -i '/"imageGCHigh*/a \ \ "imageGCLowThresholdPercent": 50,' /etc/kubernetes/kubelet/kubelet-config.json
fi

## Initializing kubelet based on spot/ondemand
instance_id=$(curl -s http://169.254.169.254/latest/meta-data/instance-id)

instance_type=$(aws ec2 describe-instances --instance-ids $instance_id --query 'Reservations[0].Instances[0].InstanceLifecycle' --output text)

export AWS_REGION=$(curl -s http://169.254.169.254/latest/dynamic/instance-identity/document | jq -r .region)

if [ "$instance_type" == "spot" ]; then

  /etc/eks/bootstrap.sh '${CLUSTER_NAME}' --b64-cluster-ca '${B64_CLUSTER_CA}' --apiserver-endpoint '${API_SERVER_URL}' --kubelet-extra-arg "--system-reserved cpu=250m,memory=0.2Gi,ephemeral-storage=1Gi --kube-reserved cpu=250m,memory=1Gi,ephemeral-storage=1Gi --eviction-hard memory.available<0.2Gi,nodefs.available<10% --allowed-unsafe-sysctls net.core.somaxconn,net.ipv4.tcp_tw_reuse --event-qps=0 --read-only-port=0"

else

  /etc/eks/bootstrap.sh '${CLUSTER_NAME}' --b64-cluster-ca '${B64_CLUSTER_CA}' --apiserver-endpoint '${API_SERVER_URL}' --kubelet-extra-arg "--system-reserved cpu=250m,memory=0.2Gi,ephemeral-storage=1Gi --kube-reserved cpu=250m,memory=1Gi,ephemeral-storage=1Gi --eviction-hard memory.available<0.2Gi,nodefs.available<10% --allowed-unsafe-sysctls net.core.somaxconn,net.ipv4.tcp_tw_reuse --event-qps=0 --read-only-port=0"

fi
  1. Local null provisioner to setup kubectl access by default for the role being used. assume kubectl is already setup

  2. Support for automatically adding spot node-group role added to auth config map for node to show up automatically show up in the cluster. Currently it has to be done manually..

  3. The cluster is opened up for public access which is not good security posture for the cluster. I made the change to support access from local runner/instance public IP to limit the access

Hi @calshankar, points 1 and 4 are definitely something nice to add.

Points 2 and 3 were something I wanted to added, however I think it is much cleaner when we don't add any local provisioners for kubectl. With regards to configmap, it will need to be added to terraform-k8s as I don't want to mix aws and kubernetes providers. I have ran into mega issues previously.