inikep / lzbench

lzbench is an in-memory benchmark of open-source LZ77/LZSS/LZMA compressors

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

lzbench gets killed on low memory devices without any error message

OTLabs opened this issue · comments

I run lzbench (master branch) on NanoPi NEO2 aarch64 board to test all zstd compression levels on Silesia compression corpus.

Everything runs fine till I get to level 22, at this point lzbench gets killed without any error message.

Running zstd on same file with compression level 22 produces error message.

Would quite appreciate if you can incorporate error message in lzbench for low memory condition.

I would quite appreciate your comments on memory requirements estimation for each compression levels also.

Here I would like to share some output logs:

otlabs@nanopineo2:~$ free
              total        used        free      shared  buff/cache   available
Mem:        1011064       80988      723160       11536      206916      846836
Swap:        505528       13564      491964

otlabs@nanopineo2:~/dev/lzbench$ zstd --ultra -22 silesia.tar
zstd: error 11 : Allocation error : not enough memory

otlabs@nanopineo2:~/dev/lzbench$ zstd --ultra -21 silesia.tar
silesia.tar     : 24.88%   (211957760 => 52743949 bytes, silesia.tar.zst) 

otlabs@nanopineo2:~/dev/lzbench$ ./lzbench -ezstd,22 silesia.tar
lzbench 1.8 (64-bit Linux)   Assembled by P.Skibinski
Compressor name         Compress. Decompress. Compr. size  Ratio Filename
memcpy                    675 MB/s   769 MB/s   211947520 100.00 silesia.tar
Killed

otlabs@nanopineo2:~/dev/lzbench$ ./lzbench -ezstd,22 lzbench18_sorted.md 
lzbench 1.8 (64-bit Linux)   Assembled by P.Skibinski
Compressor name         Compress. Decompress. Compr. size  Ratio Filename
memcpy                   3429 MB/s  4788 MB/s       14940 100.00 lzbench18_sorted.md
zstd 1.4.4 -22           0.59 MB/s   104 MB/s        3573  23.92 lzbench18_sorted.md
done... (cIters=1 dIters=1 cTime=1.0 dTime=2.0 chunkSize=1706MB cSpeed=0MB)
commented

Try the following options:
-b# set block/chunk size to # KB (default = MIN(filesize,1747626 KB))
-m# set memory limit to # MB (default = no limit)