An efficient compression tool specific for log files. It compresses log files by utilizing the inherent structures of raw log messages, and thereby achieves a high compression ratio.
- python3
- pandas
Logzip can be directly execute through source code.
-
Download and install python3 here.
-
Install Pandas.
$ pip3 install pandas -
Clone logzip.
$ clone https://github.com/logpai/logzip.git
We've conducted comprehensive experiments to evaluate the efficiency of logzip on five real-world datasets. All the datasets that we use are available at loghub.
A demo is uploaded to this repo (logzip/src/demo). We use a HDFS log file with 2k lines as a demo.
$ cd logzip/src/demo/
$ python3 zip_demo.py$ cd logzip/src/demo/
$ python3 unzip_demo.py