kube-scheduler-plugins cannot be started due to insufficient permissions.

[JustinWang]

Area

• Scheduler
• Controller
• Helm Chart
• Documents

Other components

No response

What happened?

I use this configuration example, but my kube-scheduler has an error log and cannot be started due to lack of permissions.

apiVersion: kubescheduler.config.k8s.io/v1
kind: KubeSchedulerConfiguration
leaderElection:
  leaderElect: false
clientConnection:
  kubeconfig: "/etc/kubernetes/scheduler.conf"
profiles:
  - schedulerName: network-aware-scheduler
    plugins:
      queueSort:
        enabled:
          - name: TopologicalSort
        disabled:
          - name: "*"
      preFilter:
        enabled:
          - name: NetworkOverhead
      filter:
        enabled:
          - name: NetworkOverhead
      score:
        disabled: # Preferably avoid the combination of NodeResourcesFit with NetworkOverhead
          - name: NodeResourcesFit
        enabled: # A higher weight is given to NetworkOverhead to favor allocation schemes with lower latency.
          - name: NetworkOverhead
            weight: 5
          - name: BalancedAllocation
            weight: 1
    pluginConfig:
      - name: TopologicalSort
        args:
          namespaces:
            - "default"
      - name: NetworkOverhead
        args:
          namespaces:
            - "default"
          weightsName: "UserDefined" # The respective weights to consider in the plugins
          networkTopologyName: "net-topology-test" # networkTopology CR to be used by the plugins

E0921 06:51:18.486853       1 reflector.go:140] github.com/diktyo-io/appgroup-api/pkg/generated/informers/externalversions/factory.go:119: Failed to watch *v1alpha1.AppGroup: failed to list *v1alpha1.AppGroup: appgroups.appgroup.diktyo.x-k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "appgroups" in API group "appgroup.diktyo.x-k8s.io" at the cluster scope

I have performed the following steps:

What did you expect to happen?

I hope it won't fail to start due to insufficient permissions.

How can we reproduce it (as minimally and precisely as possible)?

No response

Anything else we need to know?

No response

Kubernetes version

$ kubectl version
# paste output here

Client Version: v1.28.2
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.28.2

Scheduler Plugins version

v0.26.7

[Wei Huang]

You'd need to apply extra RBAC privileges to access the CR objects NetworkOverhead plugin rely on.

[JustinWang]

您需要应用额外的 RBAC 权限才能访问 NetworkOverhead 插件所依赖的 CR 对象。
Hi@Huang-Wei
Is there any rbac authorization I need for this project? New to this project

[JustinWang]

I used the master/manifests/networktopology/cluster-role.yaml file, and now there is no insufficient permission prompt. However, when using the BalancedAllocation plugin mentioned above, it is still not possible to start the kube-scheduler-plugins, and an error is reported that the plugin cannot be found.

[Wei Huang]

The Helm chart only enabled a couple of plugin, and NetworkOverhead is not one of them. So with NetworkOverhead/BalancedAllocation enabled, you'd need to grant RBAC privileges to system:kube-scheduler.

RBAC rules reference:

scheduler-plugins/manifests/install/all-in-one.yaml

Lines 11 to 17 in 7a5b855

 # for network-aware plugins add the following lines (scheduler-plugins v.0.24.9) 
 #- apiGroups: [ "appgroup.diktyo.k8s.io" ] 
 #  resources: [ "appgroups" ] 
 #  verbs: [ "get", "list", "watch", "create", "delete", "update", "patch" ] 
 #- apiGroups: [ "networktopology.diktyo.k8s.io" ] 
 #  resources: [ "networktopologies" ] 
 #  verbs: [ "get", "list", "watch", "create", "delete", "update", "patch" ]