lzbench gets killed on low memory devices without any error message
OTLabs opened this issue · comments
I run lzbench (master branch) on NanoPi NEO2 aarch64 board to test all zstd compression levels on Silesia compression corpus.
Everything runs fine till I get to level 22, at this point lzbench gets killed without any error message.
Running zstd on same file with compression level 22 produces error message.
Would quite appreciate if you can incorporate error message in lzbench for low memory condition.
I would quite appreciate your comments on memory requirements estimation for each compression levels also.
Here I would like to share some output logs:
otlabs@nanopineo2:~$ free
total used free shared buff/cache available
Mem: 1011064 80988 723160 11536 206916 846836
Swap: 505528 13564 491964
otlabs@nanopineo2:~/dev/lzbench$ zstd --ultra -22 silesia.tar
zstd: error 11 : Allocation error : not enough memory
otlabs@nanopineo2:~/dev/lzbench$ zstd --ultra -21 silesia.tar
silesia.tar : 24.88% (211957760 => 52743949 bytes, silesia.tar.zst)
otlabs@nanopineo2:~/dev/lzbench$ ./lzbench -ezstd,22 silesia.tar
lzbench 1.8 (64-bit Linux) Assembled by P.Skibinski
Compressor name Compress. Decompress. Compr. size Ratio Filename
memcpy 675 MB/s 769 MB/s 211947520 100.00 silesia.tar
Killed
otlabs@nanopineo2:~/dev/lzbench$ ./lzbench -ezstd,22 lzbench18_sorted.md
lzbench 1.8 (64-bit Linux) Assembled by P.Skibinski
Compressor name Compress. Decompress. Compr. size Ratio Filename
memcpy 3429 MB/s 4788 MB/s 14940 100.00 lzbench18_sorted.md
zstd 1.4.4 -22 0.59 MB/s 104 MB/s 3573 23.92 lzbench18_sorted.md
done... (cIters=1 dIters=1 cTime=1.0 dTime=2.0 chunkSize=1706MB cSpeed=0MB)
Try the following options:
-b# set block/chunk size to # KB (default = MIN(filesize,1747626 KB))
-m# set memory limit to # MB (default = no limit)