Giters

ckolivas / lrzip

Long Range Zip

Home Page:http://lrzip.kolivas.org

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Streaming issue "No space left on device" in lrzip 0.651

stevenshiau opened this issue · comments

commented

We encountered an issue when using lrzip to compress the stdin data and output to a file. It gave an error "No space left on device". However, we are very sure the disk space is enough.
It's on Debian Sid live system. If we switched to lrzip 0.641, this issue was not reproduced.
More information is provided in the following.
If you need more info, please let me know. Thanks.

root@debian:~# dpkg -l lrzip
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-==============-============-============-======================================================
ii lrzip 0.651-2 amd64 compression program with a very high compression ratio

root@debian:~# dd if=/dev/nvme0n1p3 | lrzip -v - > /home/partimag/nvme0n1p3.dd.lrz
The following options are in effect for this COMPRESSION.
Threading is ENABLED. Number of CPUs detected: 4
Detected 4119420928 bytes ram
Compression level 7
Nice Value: 19
Show Progress
Verbose
Temporary Directory set as: ./
Compression mode is: LZMA. LZ4 Compressibility testing enabled
Heuristically Computed Compression Window: 6 = 600MB
Outputting to stdout.
Minimising number of threads to 3 to limit memory usage
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Unable to compress entirely in ram, will use physical files
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Write of length 25336877 failed - No space left on device
Failed to write_buf s_buf in compthread 0
No space left on device
Fatal error - exiting

After downgraded to 0.641, no such issue.
root@debian:~# dpkg -l lrzip
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-==============-===============-============-======================================================
ii lrzip 0.641-1+deb11u1 amd64 compression program with a very high compression ratio

root@debian:~# dd if=/dev/nvme0n1p3 | lrzip -v - > /home/partimag/nvme0n1p3.dd.lrz
The following options are in effect for this COMPRESSION.
Threading is ENABLED. Number of CPUs detected: 4
Detected 4119420928 bytes ram
Compression level 7
Nice Value: 19
Show Progress
Verbose
Temporary Directory set as: ./
Compression mode is: LZMA. LZ4 Compressibility testing enabled
Heuristically Computed Compression Window: 6 = 600MB
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
Beginning rzip pre-processing phase
124370944+0 records in
124370944+0 records out
63677923328 bytes (64 GB, 59 GiB) copied, 3372.52 s, 18.9 MB/s
Compression Ratio: 0.000. Average Compression Speed: 17.930MB/s.
Total time: 00:56:26.90

root@debian:~# free -h
total used free shared buff/cache available
Mem: 3.8Gi 242Mi 2.8Gi 287Mi 788Mi 3.1Gi
Swap: 0B 0B 0B

root@debian:~# df -h
Filesystem Size Used Avail Use% Mounted on
udev 1.9G 0 1.9G 0% /dev
tmpfs 393M 2.1M 391M 1% /run
/dev/sr0 365M 365M 0 100% /run/live/medium
/dev/loop0 305M 305M 0 100% /run/live/rootfs/filesystem.squashfs
tmpfs 2.0G 286M 1.7G 15% /run/live/overlay
overlay 2.0G 286M 1.7G 15% /
tmpfs 2.0G 0 2.0G 0% /dev/shm
tmpfs 5.0M 8.0K 5.0M 1% /run/lock
tmpfs 2.0G 48K 2.0G 1% /tmp
tmpfs 393M 0 393M 0% /run/user/1000
/dev/nvme0n3p1 1000G 66G 935G 7% /home/partimag

commented

This commit may be the culprit.