Decoding data with zlib wrappers is slow
oyvindln opened this issue · comments
oyvindln commented
I suspect it may be the adler32 checksum calculation that is slow, as data with gzip or no wrapper decodes significantly faster.
There is an adler32 crate that may help here.
Takeru Ohta commented
Your point is correct. I changed to use the adler32
crate at 4af9f2a .
Below is a brief benchmark result for decoding zlib compressed data by two adler32 implementations.
$ ls -lh /tmp/zlib_data
-rw-rw-rw- 1 ohta ohta 93M Jul 18 00:40 /tmp/zlib_data
# Original implementation version of adler32 checksum
$ time -p cargo run --release --example flate -- -i /tmp/zlib_data -o /dev/null zlib-decode
real 3.11
user 2.75
sys 0.34
# `adler32` crate version
> time -p cargo run --release --example flate -- -i /tmp/zlib_data -o /dev/null zlib-decode
real 2.48
user 2.09
sys 0.32
The improvement is obvious. Thank you for your advice!