Is there no way to assign a static IP to the self node pool?
Tim-Schwalbe opened this issue · comments
When I got it right the self node pool will not get a static IP because KubeIP delete somehow 0.0.0.0 for reassign and the can not connect anymore to gcloud. I tried it and it completly make the cluster unusable.
Can someone please explain why and how this happens?
Is there is a way to assign all nodes an static IP?
The 0.0.0.0 thing doesn't break anything since it fails to remove it anyway: #85
Basically what happens is the removal of the original IP and the assignment of the new IP is a two step operation: delete the old access config and add the new access config. During this time, internet connection is unavailable from the node.
When kubeIP deletes an IP, it issues another request to wait until it is done:
kubeip/pkg/kipcompute/compute.go
Line 170 in c3dd8ec
The wait operation is terminated when the internet connection is lost (the network interface is reloaded I think, so all open connections are dropped). kubeIP sees this as an error and aborts before it can add the new IP, leaving it in the state where it is now.
At this point a new IP can't be added because of two reasons:
- Connections to the Google API requires an internet connection. This however can be worked around. You can create a VPC network that has Private Google Access enabled, and have a private DNS zone that provides the IPs for either
private.googleapis.comorrestricted.googleapis.comand CNAME*.googleapis.comto one of those. - Connection to the Kubenetes master control plane also requires an internet connection. This cannot be worked around AFAIK. You would need VPC network peering between the two networks and since the control plane is managed by Google, there's no option to do this. What kubeIP could do here is instead of allowing the error on the wait operation to abort back to the start, it could try just retry the wait request or something similar. At this point, connection to the Kubernetes control plane is not actually necessary if it remembers the state where it is at rather than aborting.
This issue only however applies to public clusters. If you create a private cluster this is not an issue because:
- There is no delete IP operation necessary
- Private Google Access is enabled and configured by default
- A VPC peering connection to the control plane exists
so no functional external internet connection is required at all for private clusters.
(yes, this is basically me dumping my personal experience here for anyone who may need it)
@Bo98 So KubeIP does not work for public clusters?
When assiging an IP to the node it's running on, no, at least not currently. If you must use a public cluster you'll need to run kubeIP on a separate node.
Private clusters were fine for my case, since you can still assign a public IP for public access to whatever's running in the container.
fixed with KubeIP v2