Make kube-proxy optional during `kubeadm upgrade apply`
zerkms opened this issue · comments
What keywords did you search in kubeadm issues before filing this one?
kube-proxy upgade
Is this a BUG REPORT or FEATURE REQUEST?
FEATURE REQUEST
Versions
kubeadm version (use kubeadm version):
kubeadm version: &version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.3", GitCommit:"2d3c76f9091b6bec110a5e63777c332469e0cba2", GitTreeState:"clean", BuildDate:"2019-08-19T11:11:18Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"linux/amd64"}
Environment:
- Kubernetes version (use
kubectl version):
Client Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.3", GitCommit:"2d3c76f9091b6bec110a5e63777c332469e0cba2", GitTreeState:"clean", BuildDate:"2019-08-19T11:13:54Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.3", GitCommit:"2d3c76f9091b6bec110a5e63777c332469e0cba2", GitTreeState:"clean", BuildDate:"2019-08-19T11:05:50Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"linux/amd64"}
- Cloud provider or hardware configuration: bare metal
- OS (e.g. from /etc/os-release): ubuntu:bionic
- Kernel (e.g.
uname -a):4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux - Others:
What happened?
At the moment kubeadm upgrade apply installs kube-proxy even if I don't have it running (I'm using kube-router), that breaks networking on the nodes.
I checked source code and from there it looks PerformPostUpgradeTasks runs proxy.EnsureProxyAddon unconditionally.
What you expected to happen?
There should be a key or any other way to avoid installing it.
How to reproduce it (as minimally and precisely as possible)?
Create an older cluster, remove kube-proxy, upgrade it.
Anything else we need to know?
this was reported and discussed multiple times already.
There should be a key or any other way to avoid installing it.
the last good proposal was to check if the kube-proxy config map exists before calling proxy.EnsureProxyAddon on upgrade and possibly skip it.
/assign
given we are entering code freeze for 1.16, not sure we should send a PR for this.
it falls under feature changes and possibly we need to vote for it during kubeadm office hours.
it falls under feature changes
It breaks working clusters. It's a severe bug.
kubeadm 1.16 and older version still consider kube-proxy as critical component.
without kube-proxy your cluster is currently unsupported.
Hi @neolit123 , I will work on this issue after kubeadm team have conclusion.
As per comment on kubernetes/kubernetes#82248
I don't want to block on this, but as per comment in the issue, I'm in favour of a different approach that makes explicit what the user want instead of guessing cluster condition/ignoring arbitrary errors.
The approach I'm suggesting requires API changes, but we are at the beginning of a new cycle and there is room to make this effort enabler/coordinated with the addon project integration, which is the long term answer to the issue this PR aims to fix
/kind api-change
@rosti, I would like to work on this issue. Could you guide me on it? Thank you.
does this ticket need the api-change label?
currently we can check for the lack of a kube-proxy CM during upgrade and skip the re-apply of the addon. tracking the api-change for optional addons should be another ticket?
Discussed in kubeadm Office Hours (Oct 2)
"Make AddOns opt-in by supplying an array of names in a field of the ClusterConfiguration"
See the document for brainstorming on API examples.
When the user supplies a custom KubeProxyConfiguration and the addon is disabled, we should print a warning -- similar warnings may apply for CoreDNS
this will be resolved once #1318 is resolved.
please watch that ticket.