README

A small benchmark suite for testing the compression / decompression speed and compression ratios of different versions of zlib.

Usage

Running the benchmark

Run the benchmark with a command like the following:

perl bench.pl --output-format=json --output-file=results.json

This will store the results in a json file for later analysis.

Pretty-print the results

To pretty-print the results of an earlier run stored in a json file, use the --read-json flag.

perl bench.pl --read-json=results.json

Changing which versions are tested against

To change the versions which are tested against, you need to edit the @versions variable in the source code (either the git repository urls or the version hashes). Note that if you change the definition of existing entries under versions (or if you e.g. upgrade the compiler), you’ll probably want to run with the --recompile flag the next time.

Adding new input files to the benchmark

Any files starting with a small letter in the corpus/ directory will be used as inputs, each one creating a new benchmark family (decompression, compression at each specified compression level). The name of the file is used as the benchmark id in reports.

Full options

--help                 Print a help message
--compress-iters=...   Number of times each file is compressed in one
                       benchmark run
--compress-levels=...  Comma-separated list of compression levels to use
--decompress-iters=... Number of times each file is compressed in one
                       benchmark run
--output-file=...      File (- for stdout) where results are printed to
--output-format=...    Format to output results in (pretty, json)
--read-json=...        Don't run benchmarks, but read results from this file
--recompile            If passed, recompile all zlib versions before test
--runs=...             Number of runs for each benchmark
--quiet                Don't print progress reports to STDERR

Example output

                     baseline               cloudflare             intel                  
compress executable -5 (x 10)
  Compression ratio:  0.33                   0.33                   0.34                  
 Execution time [s]:  2.66 ± 0.02 (100.00%)  2.17 ± 0.02 ( 81.70%)  1.49 ± 0.01 ( 56.07%) 
compress html -5 (x 10)
  Compression ratio:  0.34                   0.34                   0.33                  
 Execution time [s]:  1.62 ± 0.01 (100.00%)  1.31 ± 0.02 ( 81.18%)  0.85 ± 0.01 ( 52.81%) 
compress jpeg -5 (x 10)
  Compression ratio:  1.00                   1.00                   1.00                  
 Execution time [s]:  0.93 ± 0.01 (100.00%)  0.91 ± 0.01 ( 97.45%)  1.08 ± 0.01 (115.38%) 
decompress executable (x 50)
 Execution time [s]:  1.96 ± 0.00 (100.00%)  1.96 ± 0.01 (100.17%)  1.93 ± 0.01 ( 98.56%) 
decompress html (x 50)
 Execution time [s]:  1.13 ± 0.00 (100.00%)  1.13 ± 0.00 ( 99.96%)  1.11 ± 0.01 ( 97.56%) 
decompress jpeg (x 50)
 Execution time [s]:  0.29 ± 0.00 (100.00%)  0.29 ± 0.00 (102.03%)  0.29 ± 0.00 (103.15%)

tuananh / zlib-bench