Cloud controllers should not reuse names and cluster roles of KCM controllers
nckturner opened this issue · comments
Currently, a cluster operator can run their external cloud controller manager with --use-service-account-credentials
enabled or disabled. If the flag is enabled, then each controller (control loop) gets its own service account token, using a hardcoded name (like "node-controller"). This is the name used for the service account, so it also needs to be present in the role binding for that controller. Currently, cloud providers may not have taken the time to recreate each controllers role and rolebinding separately from the upstream roles, and are (at least in AWS's case) piggybacking on the upstream roles. We want to move to custom cluster roles and role bindings, and it might make sense to deprecate the upstream roles eventually to reduce confusion.
I propose we differentiate the names of the controllers once they move to the external CCM, so that they can be differentiated in audit logs and from similar loops in the KCM. (e.g., the "node-controller" currently refers to three separate controllers, the cloud-node, the cloud-node-lifecycle, and the in-tree node-lifecycle (and maybe node-ipam too :D).
If we decided to do this, we could allow these names to be plumbed in from each cloud provider repository, so they can be cloud provider specific, or we could all agree on something like cloud-*-controller.
I agree with the premise, but concerned about breaking compatibility
I agree with the premise, but concerned about breaking compatibility
Definitely, but regardless of whether the upstream roles are deprecated, we could allow cloud providers to both use the k8s.io/cloud-provider libraries and use custom names and roles, right?
What about extending https://github.com/kubernetes/kubernetes/blob/master/staging/src/k8s.io/cloud-provider/app/controllermanager.go#L360?
Right now its just the controller name and initializer. However we could extend that to include the desired SA.
We already allow the cloud providers to over-ride this, so it seems a natural way to let them control the change.
Yeah, that seems like a good place to do it.
The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.
This bot triages issues and PRs according to the following rules:
- After 90d of inactivity,
lifecycle/stale
is applied - After 30d of inactivity since
lifecycle/stale
was applied,lifecycle/rotten
is applied - After 30d of inactivity since
lifecycle/rotten
was applied, the issue is closed
You can:
- Mark this issue or PR as fresh with
/remove-lifecycle stale
- Mark this issue or PR as rotten with
/lifecycle rotten
- Close this issue or PR with
/close
- Offer to help out with Issue Triage
Please send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle stale