k8s node can't mount pvc until node restarted
morgoved opened this issue · comments
2024-05-17T04:03:30.818498066Z stderr F E0517 04:03:30.818193 1 utils.go:81] GRPC error: rpc error: code = Internal desc = failed to stat file /var/lib/kubelet/pods/8dff9f0d-7f82-415c-9a5b-c32e81fb3ea4/volumes/kubernetes.iocsi/pvc-a06f67fa-953d-4136-bec3-363c4ea5f687/mount: lstat /var/lib/kubelet/pods/8dff9f0d-7f82-415c-9a5b-c32e81fb3ea4/volumes/kubernetes.iocsi/pvc-a06f67fa-953d-4136-bec3-363c4ea5f687/mount: host is down
What happened:
In one day service said that we can get files from pvc.
after restart i see this into csi logs.
Only atfer that i restarted kubernetes nodes and all worked.
- CSI Driver version: v1.14.0
- Kubernetes version (use
kubectl version): 1.29.4 - OS (e.g. from /etc/os-release): VERSION="22.04.4 LTS (Jammy Jellyfish)"
- Kernel (e.g.
uname -a): 5.15.0-107-generic - Install tools: helm
- Others: k8s installed on kvm vms by kubeadm, cni - calico