What Is This
This is a collection of software that can test the compression efficiency of various lossy compressed image formats, according to various quality metrics. It can create graphs of the test data.
Supported Formats
- JPEG (via libjpeg-turbo)
- JPEG (via mozjpeg)
- WebP
- HEVC-MSP
- JPEG XR
Supported Quality Metrics
- PSNR-HVS-M (luma-only)
- Y-SSIM (luma-only SSIM)
- RGB-SSIM (average of SSIM applied to R, G, and B channels)
- MS-SSIM (luma-only)
Supported Operating Systems
This test suite is developed and tested on Linux (Ubuntu 16.04) and OS X (10.11). It's probably easy to get things working on other versions of Linux and OS X. Little to no effort has gone into getting this working on Windows.
Prerequisites
- ImageMagick, specifically the 'convert' utility
- http://www.imagemagick.org/
- Version 6.8.6 or higher required, earlier versions have a bug in YUV conversion.
- Note that latest Ubuntu stable still has buggy older version. Fedora 20+ has a good version.
- python
- Any version > 2.7.0 and < 3.0 is fine.
- gnuplot
- http://www.gnuplot.info/
- Any version > 4.6.0
- libjpeg-turbo
- http://libjpeg-turbo.virtualgl.org/
- We prefer libjpeg-turbo over IJG libjpeg
- Tested with 1.5.0 release
- Place directory called "libjpeg-turbo-1.5.0" alongside this test suite.
- mozjpeg
- git clone https://github.com/mozilla/mozjpeg.git
- Tested with latest git revision
- Place directory called "mozjpeg" alongside this test suite.
- HEVC-MSP Encoder/Decoder
- svn checkout http://hevc.hhi.fraunhofer.de/svn/svn_HEVCSoftware/
- Tested with r3928
- Place directory called "svn_HEVCSoftware" alongside this test suite.
- JPEG XR Encoding/Decoding Library
- git clone https://git01.codeplex.com/jxrlib
- Tested with git revision db41362ccbc7
- Place directory called "jxrlib" alongside this test suite.
- WebP Encoding/Decoding Library
- https://developers.google.com/speed/webp/download
- Tested with 0.5.1 release
- Place directory called "libwebp-0.5.1" alongside this test suite.
Usage
Common Requirements
- Install required software
- Build included encoder/decoder wrappers and metrics programs
- OS X: $ ./build-osx.sh
- Linux: $ ./build-linux.sh
Manual Operation
- Collect data for desired format and images
- $ ./rd_collect.py jpeg images/kodak/*.png
- $ ./rd_collect.py mozjpeg images/kodak/*.png
- Average data from images
- $ ./rd_average.py images/kodak/*.jpeg.out > jpeg.out
- $ ./rd_average.py images/kodak/*.mozjpeg.out > mozjpeg.out
- Generate graphs
- $ ./rd_plot.py Kodak jpeg.out mozjpeg.out
Full Automatic Run
- $ ./run_all.py
- Note that this can take a long time to complete, ~9 hours with a 6-core (12 w/hyperthreading) Intel i7 6850k with 32GB RAM and NVRAM disk.