部署node的kubelet时报错：k8s.io/kubernetes/pkg/kubelet/kubelet.go:422: Failed to list *v1.Node: nodes is forbidden: User "system:node:172.21.24.251" cannot list nodes at the cluster scope
Vortexxxx opened this issue · comments
严格按照教程一步步来的
master和node都在同一台机器上，master部署已经部署好，部署node的时候报出如下错误：
Nov 04 17:58:03 node-2 kubelet[28185]: E1104 17:58:03.101010 28185 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod: pods is forbidden: User "system:node:172.21.24.251" cannot list pods at the cluster scope
Nov 04 17:58:03 node-2 kubelet[28185]: E1104 17:58:03.101903 28185 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:422: Failed to list *v1.Node: nodes is forbidden: User "system:node:172.21.24.251" cannot list nodes at the cluster scope
刚开始没有发现这个错误，正常执行kubectl get csr得到以下结果：
NAME AGE REQUESTOR CONDITION
node-csr-P0zjqO2H6y4IO1O-ShoZEFvgeZ_C1PptCZ1aSe7ada4 2h kubelet-bootstrap Approved,Issued
node-csr-Q3cnW1eRmgu0Ttica8-gHmixcnkQbo9iDNuX0CH7zgU 1h kubelet-bootstrap Approved,Issued
node-csr-udzBibfgWpeRcOFgxGlM4_jTcH7J-9gcuplrDZUTqi4 2h kubelet-bootstrap Approved,Issued
全部都已经执行通过证书，之所以有三次是别的机器的申请，可以先暂时忽略
kubelet 的service配置文件：
[Unit]
Description=Kubernetes Kubelet Server
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
After=docker.service
Requires=docker.service
[Service]
WorkingDirectory=/var/lib/kubelet
EnvironmentFile=-/etc/kubernetes/config
EnvironmentFile=-/etc/kubernetes/kubelet
ExecStart=/usr/local/bin/kubelet \
$KUBE_LOGTOSTDERR \
$KUBE_LOG_LEVEL \
$KUBELET_API_SERVER \
$KUBELET_ADDRESS \
$KUBELET_PORT \
$KUBELET_HOSTNAME \
$KUBE_ALLOW_PRIV \
$KUBELET_POD_INFRA_CONTAINER \
$KUBELET_ARGS
Restart=on-failure
[Install]
WantedBy=multi-user.target
kubelet的配置文件/etc/kubernetes/kubelet
###
## kubernetes kubelet (minion) config
#
## The address for the info server to serve on (set to 0.0.0.0 or "" for all interfaces)
KUBELET_ADDRESS="--address=172.21.24.251"
#
## The port for the info server to serve on
#KUBELET_PORT="--port=10250"
#
## You may leave this blank to use the actual hostname
#KUBELET_HOSTNAME="--hostname-override=172.21.24.251"
#
## location of the api-server
#KUBELET_API_SERVER="--api-servers=http://172.21.24.251:8080"
#
## pod infrastructure container
#KUBELET_POD_INFRA_CONTAINER="--pod-infra-container-image=sz-pg-oam-docker-hub-001.tendcloud.com/library/pod-infrastructure:rhel7"
#
## Add your own!
KUBELET_ARGS="--cgroup-driver=cgroupfs --experimental-bootstrap-kubeconfig=/etc/kubernetes/bootstrap.kubeconfig --kubeconfig=/etc/kubernetes/kubelet.kubeconfig --require-kubeconfig --cert-dir=/etc/kubernetes/ssl --cluster-domain=cluster.local --hairpin-mode promiscuous-bridge --serialize-image-pulls=false"
/etc/kubernetes/bootstrap.kubeconfig配置文件
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUR2akNDQXFhZ0F3SUJBZ0lVYnN6cyszNVJOTG5ldEE4Znk0d1FnazVJa0lVd0RRWUpLb1pJaHZjTkFRRUwKQlFBd1pURUxNQWtHQTFVRUJoTUNRMDR4RURBT0JnTlZCQWdUQjBKbGFVcHBibWN4RURBT0JnTlZCQWNUQjBKbAphVXBwYm1jeEREQUtCZ05WQkFvVEEyczRjekVQTUEwR0ExVUVDeE1HVTNsemRHVnRNUk13RVFZRFZRUURFd3ByCmRXSmxjbTVsZEdWek1CNFhEVEUzTVRFd05EQTFNell3TUZvWERUSXlNVEV3TXpBMU16WXdNRm93WlRFTE1Ba0cKQTFVRUJoTUNRMDR4RURBT0JnTlZCQWdUQjBKbGFVcHBibWN4RURBT0JnTlZCQWNUQjBKbGFVcHBibWN4RERBSwpCZ05WQkFvVEEyczRjekVQTUEwR0ExVUVDeE1HVTNsemRHVnRNUk13RVFZRFZRUURFd3ByZFdKbGNtNWxkR1Z6Ck1JSUJJakFOQmdrcWhraUc5dzBCQVFFRkFBT0NBUThBTUlJQkNnS0NBUUVBcmlZSnpXUFVVaC9Wc0FPMkhMY0QKWmY4ZWc4NEN6S3RtNlZpTytjTzNFZ0djcEJIa3hZMjRsT3dONHdjamk0NE90Z3UrV0YzS3I2M3g4RmZxb1VJNQo1cFRnV2d0cXFTbHpUUWdNK3FKTzFCMWZjTHdtamFqeFlGbFBOTDZOWkFnYU8vRDU4Tkp6dTVkVHo2VE8rNXVLCk55ZERtVk91eFM3VWtpVkovU3FiLzBhbjBZeEREWW5YV29mVnB5R0pHRU5qMkcySDhqWjVMay9ReTFQVlBUencKUmZiaDhSZXJWa3paT1FmRFI0YVA0T250L0xodzlFUFlmQ2RGd3RkN0c0Q3ZXSm96Z0Y1WDljanVMWWpkRzNucQpOb3A5clRrN0p2OVgwV1lyOHFzZUlhL1JuSHpoMGUvaDBFSzhmT2w0NVRPSDJlNHBIMnlVLzBLOEFVMUh3bHhsClZRSURBUUFCbzJZd1pEQU9CZ05WSFE4QkFmOEVCQU1DQVFZd0VnWURWUjBUQVFIL0JBZ3dCZ0VCL3dJQkFqQWQKQmdOVkhRNEVGZ1FVRmIwZ0s0ZWN1YUVCZ0VPUCtEejN0UUFjb2Vrd0h3WURWUjBqQkJnd0ZvQVVGYjBnSzRlYwp1YUVCZ0VPUCtEejN0UUFjb2Vrd0RRWUpLb1pJaHZjTkFRRUxCUUFEZ2dFQkFBNSs3dUV6VWlIZ3BFTThmQzRwCjY4OXcrK29vRXBST1lsOEtmS1BBL1lCS2c5YkU2YWRnMGxlVzdzcFRlQVF0cmlZMFlsY2N5NWQwdHdzYllFL3UKQklJWXN4aEdvaUtocXR0TTNNRzZ6VmxvWitrRTFjaWg2ZDBLWWpJWFo0MkxVL1pRSGtIMWh3cDlkR0ZqcjhyTwpaMHo3NWxVNVZFWGU5S3ZsTmJuQXRxak1FSlB3SVVrWWlSTXhSVkhKMC9aWFRXTlBFYlVsanhncHQyd0lPYzhBCmcvZ3NaaUhhRVdHZ3g1QjFqbXZCd1d4dUpFQ2pMSjRQSzl1NU1QSlkvM05vVTlTdk9ROWNiMDluc2kyQUZleVQKZ0syVURRUVNLY1NweWtaQ2MzcElJSFZOVGhkVHlGZlQxdDQ0b05EWnRadXc2WjNTRWdPUWh0dEQ2S0VISFJ1Ygp5NGM9Ci0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K
server: https://172.21.24.251:6443
name: kubernetes
contexts:
- context:
cluster: kubernetes
user: kubelet-bootstrap
name: default
current-context: default
kind: Config
preferences: {}
users:
- name: kubelet-bootstrap
user:
as-user-extra: {}
token: 3449a2d2ce97647056e87f3020cb0a5e
如需补充什么信息请告诉我，谢谢，卡在这儿一下午了
暂时没有找到啥原因，后来全部从头来过就没有碰到这个问题了
看起来像是RBAC ClusterRole/ClusterRoleBinding规则初始化有问题，如果再碰到的话，可以检查这俩是不是有问题。
我也遇见了这个问题，非主 master 节点，重启(reboot)后与 master 建联正常
func (kl *Kubelet) GetNode() (*v1.Node, error) {
if kl.kubeClient == nil {
return kl.initialNode(context.TODO())
}
return kl.nodeLister.Get(string(kl.nodeName))
}
pkg/kubelet/kubelet_getters.go 234行
这里获取 contex.TODO是还没写吗？