Failed to launch workers while sync data from a S3Compatible storage to another
younger8586 opened this issue · comments
What happened:
(1) exe. command:
juicesync --worker 192.168.12.14 --threads 50 --no-https s3:/ABCD:EFGH@juice./test/ s3:/ABCD:EFGH@juice./test1/
(2) results:
2022/12/13 22:33:59.297285 juicefs[3272093] : Syncing from s3://juice/test/ to s3://juice/test3/ [sync.go:619]
2022/12/13 22:33:59.900130 juicefs[3272093] : Listen at 192.168.12.10:46817 [cluster.go:195]
2022/12/13 22:34:00.103541 juicefs[3272093] : launch a worker on 192.168.12.14 [cluster.go:270]
Scanned objects count: 0 / 2000 [--------------------------------------------------------------] 0 % 0s
Copied objects count: 0 0.00/s ⠧
Scanned objects count: 8547 / 8547 [==============================================================] done
Copied objects count: 8547 26.54/s
Copied objects bytes: 375.96 GiB (403679646720 Bytes) 1.17 GiB/s
Checked objects bytes: 0.00 b (0 Bytes) 0.00 b/s
Deleted objects count: 0 0.00/s
Skipped objects count: 0 0.00/s
Failed objects count: 0 0.00/s
2022/12/13 22:39:21.331453 juicefs[3272093] : Found: 8547, copied: 8547 (375.96 GiB), checked: 0 B, deleted: 0, skipped: 0, failed: 0 [sync.go:892]
There are some 'juicesync' threads on 192.168.12.10, but no 'juicesync' threads on 192.168.12.14。
The worker does not work。
What you expected to happen:
distributed sync
How to reproduce it (as minimally and precisely as possible):
Anything else we need to know?
Environment:
- JuiceFS version (use
juicefs --version) or Hadoop Java SDK version: v1.0.2 - Cloud provider or hardware configuration running JuiceFS: S3Compatible storage
- OS (e.g
cat /etc/os-release): centos7.9 - Kernel (e.g.
uname -a): 4.18.0-348.7.1.el8 - Object storage (cloud provider and region, or self maintained):S3Compatible storage
- Metadata engine info (version, cloud provider managed or self maintained):no
- Network connectivity (JuiceFS to metadata engine, JuiceFS to object storage):no
- Others:no
If parameter --verbose is added in command, 'juicesync' threads can run on these nodes (workers)。
It seems that you are using the juicesync repo. We recommend juicefs sync for now.
Can you try juicefs sync to see if this problem also exists?
juicefs sync doc: https://juicefs.com/docs/community/guide/sync/
It seems that you are using the juicesync repo. We recommend juicefs sync for now. Can you try juicefs sync to see if this problem also exists? juicefs sync doc: https://juicefs.com/docs/community/guide/sync/
yeah, juicefs sync is ok, no problem.
The bug should be only in juicesync.