A PyTorch reimplemetation for the paper:
DVC: An End-to-end Deep Video Compression Framework, Guo Lu, Wanli Ouyang, Dong Xu, Xiaoyun Zhang, Chunlei Cai, Zhiyong Gao, CVPR 2019 (Oral). [arXiv]
- Python==3.6.2
- PyTorch==1.7.1
- CompressAI==1.1.6
Vimeo90k is used for training, you can download it online and put the video sequence in ./data/vimeo90k/sequences
.
You can simply perform the most basic training with the following commands
python main_dvc.py --config DVC_1024.json
If you want to change the training settings, you can modify the corresponding json file.
The HEVC test sequence is used to test the video compression performance, you can download it and put it in ./data/h265_compress/videos/
. The test process here is based on the class B in the HEVC test sequence. You can test other common datasets such as UVG in a similar way.
The video processed by DVC has a resolution limit, and here you need to manually crop the video to a multiple of 64. Take the class B in the HEVC test sequence as an example, you need to crop 1920x1080 to 1920x1024.
cd data/h265_compress/
ffmpeg -pix_fmt yuv420p -s 1920x1080 -i ./videos/xxxx.yuv -vf crop=1920:1024:0:0 ./videos_crop/xxxx.yuv
Split the yuv video file into frames.
python convert.py
Create I frames. We need to create I frames by H.265 with QP of 22,27,32,37. The setting of the compressed I frame affects the compression performance. According to this setting (set QP as 22,27,32,37), the results obtained when testing is consistent with the DVC paper.
cd CreateI
sh h265.sh $qp 1920 1024
After finished the generating of I frames of each QP, you need to use bpps of each video in result.txt
to fill the bpps in Class HEVCDataSet
in dataset.py
.
Set the path of the trained model in the main_dvc.py
file, and enable args.hevctest
.
python main_dvc.py --config test_1024.json
You can edit the net_dvc.py
file. The specific components that build it are in the subnet
folder. Only forward
is executed during training, and compress
and decompress
need to be executed during testing. Make sure that the two parts correspond.
If you find this paper useful, please cite:
@article{lu2018dvc,
title={DVC: An End-to-end Deep Video Compression Framework},
author={Lu, Guo and Ouyang, Wanli and Xu, Dong and Zhang, Xiaoyun and Cai, Chunlei and Gao, Zhiyong},
journal={arXiv preprint arXiv:1812.00101},
year={2018}
}
Most of the code comes from PyTorchVideoCompression. Thanks to ZhihaoHu for the open source code. I replaced the entropy model used in it with the EntropyBottleneck
class in CompressAI, and performed the actual compression and decompression during testing.