estafette / estafette-gke-preemptible-killer

Kubernetes controller to spread preemption for preemtible VMs in GKE to avoid mass deletion after 24 hours

Home Page:https://helm.estafette.io

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Question: How to select the node pool?

bakayolo opened this issue · comments

Hey guys,

Thanks for your work, very helpful.
I am very new to your preemptible-killer, I have been using Spotinst until now but wondering if I cannot do something on my own.

Here are 2 questions:

  • How can I select the node pool I want the killer to work on (assuming I have more than 1 preemptible node pool)?
  • Does the killer guarantee that 2 nodes can never be down at the same time? What if the killing of a node is due to a system event?

Thanks a lot.

  1. selecting node pool is not supported yet, the code is processing ALL preemptible nodes attached to the cluster, and there is no way to limit it even via taints or annotations
  2. no, ensure your have PDB and enough of replicas, so for better safety just use non-preemptible nodes in different zones

So treat it more like a specific chaos tool. What it does it just adding annotation about random time when the node should be killed. This way when you spawn 20 nodes then they get after a moment random time to be killed, otherwise GCP could just schedule all of them to be killed exactly after 24h.

So instead of sudden abrupt large disruption you increase the chances to have many small ones. Still, GCP can trigger large dstruption because the way preemptible instances are managed.

OK well noted.
Thanks for the clear answer. :)

it seems that random termination fixed using labels, isn't it? #40