bulk sync to node <some-string> failed
YuryHrytsuk opened this issue · comments
What does <some-string>
defines? I was never able to find a node with id equal to <some-string>
. How to map <some-string>
to an actual node in my cluster?
Example log from journalctl
Feb 12 13:19:47 my-srv01 dockerd[2615989]: time="2024-02-12T13:19:47.878730800Z" level=warning msg="bulk sync to node 993b5ed1c755 failed: failed to send a TCP message during bulk sync
docker info
Client:
Context: default
Debug Mode: false
Plugins:
buildx: Docker Buildx (Docker Inc.)
Version: v0.10.2
Path: /usr/libexec/docker/cli-plugins/docker-buildx
compose: Docker Compose (Docker Inc.)
Version: v2.16.0
Path: /usr/libexec/docker/cli-plugins/docker-compose
scan: Docker Scan (Docker Inc.)
Version: v0.23.0
Path: /usr/libexec/docker/cli-plugins/docker-scan
Server:
Containers: 10
Running: 7
Paused: 0
Stopped: 3
Images: 9
Server Version: 20.10.14
Storage Driver: overlay2
Backing Filesystem: xfs
Supports d_type: true
Native Overlay Diff: true
userxattr: false
Logging Driver: gelf
Cgroup Driver: cgroupfs
Cgroup Version: 1
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: active
NodeID: gn2rsb5a806r0u3t33b1qei2f
Is Manager: false
Node Address: 10.0.3.105
Manager Addresses:
10.0.3.45:2377
Runtimes: nvidia runc io.containerd.runc.v2 io.containerd.runtime.v1.linux
Default Runtime: nvidia
Init Binary: docker-init
containerd version: 2456e983eb9e37e47538f59ea18f2043c9a73640
runc version: v1.1.4-0-g5fd4c4d
init version: de40ad0
Security Options:
apparmor
seccomp
Profile: default
Kernel Version: 5.15.0-1030-aws
Operating System: Ubuntu 20.04.2 LTS
OSType: linux
Architecture: x86_64
CPUs: 16
Total Memory: 62.02GiB
Name: ip-10-0-3-105
ID: I5Q6:MIY6:MCQD:4FXN:OBSB:GMUQ:E5ZO:A7WX:VSJK:B7EW:VA4L:33GZ
Docker Root Dir: /mnt/docker/data
Debug Mode: true
File Descriptors: 137
Goroutines: 195
System Time: 2024-02-15T13:32:30.121115095Z
EventsListeners: 3
Registry: https://index.docker.io/v1/
Experimental: true
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
Default Address Pools:
Base: 172.17.0.0/12, Size: 20
Base: 192.168.0.0/16, Size: 24
This log is probably generated by this line: https://github.com/moby/libnetwork/blob/3797618f9a38372e8107d8c06f6ae199e1133ae8/networkdb/cluster.go#L706
This answer explain how to map <some-string>
to a particular swarm cluster Node. It would be nice to have it documented somewhere.