README.org

#+TITLE: README #+AUTHOR: Juho Snellman #+EMAIL: jsnell@iki.fi

A small benchmark suite for testing the compression / decompression speed and compression ratios of different versions of zlib.

** Usage

*** Running the benchmark

Run the benchmark with a command like the following:

#+BEGIN_EXAMPLE perl bench.pl --output-format=json --output-file=results.json #+END_EXAMPLE

This will store the results in a json file for later analysis.

*** Pretty-print the results

To pretty-print the results of an earlier run stored in a json file, use the =--read-json= flag.

#+BEGIN_EXAMPLE perl bench.pl --read-json=results.json #+END_EXAMPLE

*** Changing which versions are tested against

To change the versions which are tested against, you need to edit the =@versions= variable in the source code (either the git repository urls or the version hashes). Note that if you change the definition of existing entries under versions (or if you e.g. upgrade the compiler), you'll probably want to run with the =--recompile= flag the next time.

*** Adding new input files to the benchmark

Any files starting with a small letter in the corpus/ directory will be used as inputs, each one creating a new benchmark family (decompression, compression at each specified compression level). The name of the file is used as the benchmark id in reports.

*** Full options

#+BEGIN_EXAMPLE --help Print a help message --compress-iters=... Number of times each file is compressed in one benchmark run --compress-levels=... Comma-separated list of compression levels to use --decompress-iters=... Number of times each file is compressed in one benchmark run --output-file=... File (- for stdout) where results are printed to --output-format=... Format to output results in (pretty, json) --read-json=... Don't run benchmarks, but read results from this file --recompile If passed, recompile all zlib versions before test --runs=... Number of runs for each benchmark --quiet Don't print progress reports to STDERR #+END_EXAMPLE

** Example output

#+BEGIN_EXAMPLE baseline cloudflare intel
compress executable -5 (x 10) Compression ratio: 0.33 0.33 0.34
Execution time [s]: 2.66 ± 0.02 (100.00%) 2.17 ± 0.02 ( 81.70%) 1.49 ± 0.01 ( 56.07%) compress html -5 (x 10) Compression ratio: 0.34 0.34 0.33
Execution time [s]: 1.62 ± 0.01 (100.00%) 1.31 ± 0.02 ( 81.18%) 0.85 ± 0.01 ( 52.81%) compress jpeg -5 (x 10) Compression ratio: 1.00 1.00 1.00
Execution time [s]: 0.93 ± 0.01 (100.00%) 0.91 ± 0.01 ( 97.45%) 1.08 ± 0.01 (115.38%) decompress executable (x 50) Execution time [s]: 1.96 ± 0.00 (100.00%) 1.96 ± 0.01 (100.17%) 1.93 ± 0.01 ( 98.56%) decompress html (x 50) Execution time [s]: 1.13 ± 0.00 (100.00%) 1.13 ± 0.00 ( 99.96%) 1.11 ± 0.01 ( 97.56%) decompress jpeg (x 50) Execution time [s]: 0.29 ± 0.00 (100.00%) 0.29 ± 0.00 (102.03%) 0.29 ± 0.00 (103.15%) #+END_EXAMPLE