MacHu-GWU / compress-project

All in one data compression library.

---

Welcome to compress Documentation

There's lots of mature data compression algorithm you can choose from, compress provides normalized API to use them and switch between them.

It supports:

From Python Standard library:

• zlib.
• bz2.
• lzma, high compression ratio but slow

From Community (Additional Library Required):

• snappy, from Google, lower compression ratio but super fast! (on MacOS, you need to install it via brew install snappy, on Ubuntu, you need sudo apt-get install libsnappy-dev.
• lz4, lower ratio, super fast!

Note

some package are not installed along with compress. Because all of them needs C compiler, you have to manually install them. If you have trouble installing C compiler for your OS, read THIS TUTORIAL.

Usage:

>>> from compress import Compressor
>>> binary_data = ("hello world! " * 100).encode("utf-8")
>>> c = Compressor()
>>> c.use_gzip() # or use_bz2, use_lzma, use_lz4, use_snappy
>>> c.compress(binary_data, zlib_level=9)
>>> c.decompress(binary_data)

Other API for lazy developer:

>>> import compress
>>> compress.compress_bytes_to_bytes
>>> compress.compress_str_to_bytes
>>> compress.compress_bytes_to_b64str # compress, and returns b64 encoded str
>>> compress.compress_str_to_b64str # compress string and returns b64 encoded str

>>> compress.decompress_bytes_to_bytes # inverse of compress_bytes_to_bytes
>>> compress.decompress_bytes_to_str # inverse of compress_str_to_bytes
>>> compress.decompress_b64str_to_bytes # inverse of compress_bytes_to_b64str
>>> compress.decompress_b64str_to_str # inverse of compress_str_to_b64str

    compress_bytes_to_bytes, compress_str_to_bytes,
compress_bytes_to_b64str, compress_str_to_b64str,
decompress_bytes_to_bytes, decompress_bytes_to_str,
decompress_b64str_to_bytes, decompress_b64str_to_str,

This website provides comprehensive comparison and visualization. But how do you know how it works on your own production environment?.

compress comes with a tool to run benchmark test for All test case, All algorithm, All parameters, and you will get informative stats about ratio, compress/decompress speed in .tab and ascii table format. Then You are able to visualize it in the way you preferred.

To run benchmark test, just:

$ pip install -r requirements-benchmark.txt
$ python ./benchmark/run.py

Install

compress is released on PyPI, so all you need is:

$ pip install compress

To upgrade to latest version:

$ pip install --upgrade compress