kubeadm upgrade should not overwrite --bind-address in kube-apiserver.yaml

[Ronan LE MEILLAT]

BUG REPORT

Versions

kubeadm version 1.29.1
Environment:

• Kubernetes version: 1.29.1
• Cloud provider or hardware configuration:
• OS (e.g. from /etc/os-release): Ubuntu 22.04.4 LTS
• Kernel (e.g. uname -a):Linux ___ 5.15.0-94-generic Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
• Container runtime (CRI): containerd
• CNI: Cilium 1.15.1
• Others: "(very) Poor man HA cluster with haproxy loadbalancer to the 2 master nodes installed and listening on 127.43.43.43:6443 on each worker node and each control plane.

Bug Report

Title: Kubeadm upgrade from v1.29.1 to v1.29.2 removes --bind-address=[ip] causing port conflict

Description:

I've encountered an issue with the kubeadm upgrade from v1.29.1 to v1.29.2. I have built a cluster in a mode I've termed "poor man's high availability cluster". I've documented the entire process in this StackOverflow post.

The problem arises when the upgrade process removes the --bind-address=[ip] line on my two master nodes. This should not happen as it creates a port listening conflict. The load balancer was already listening on 127.43.43.43:6443, which prevents the kubelet from listening on 0.0.0.0:6443.

Expected Behavior:

The kubeadm upgrade should not change the listening port if it has been manually specified.

Steps to Reproduce:

1. Set up a v1.29.1 cluster as described in the StackOverflow post.
2. Perform a kubeadm upgrade from v1.29.1 to v1.29.2.

Additional Information:

This issue causes a disruption in the cluster operation and needs to be addressed to ensure smooth upgrades in the future.

Thank you for your attention to this matter.

[Lubomir I. Ivanov]

kubeadm generates new manifests from scratch when it does upgrade based on spec found in the kube-config CM. there is no plans to change that, so any custom change will be overwritten on per-node bases including custom flag values.

to support per node customization (on init, join, upgrade) you can use patches.
https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/control-plane-flags/#patches
yes, for flags they can be a bit tricky to apply, but you can leverage the fact that a component can have many --bind-address... flags and only the last one is used...so just add a new flag at the end with the patch.

create a patch file kube-apiserver+json.json and in there you can have something like:

[
    {
        "op": "add",
        "path": "/spec/containers/0/command/-",
        "value":  [ "--bind-address=SOME_ADDR" ]
    }
]

https://datatracker.ietf.org/doc/html/rfc6902#section-4.1

/kind support

[Lubomir I. Ivanov]

so once upgrade is finished if you have provided --patches with the right dir that has this file your apiserver should have the extra --bind... flag.