This repository contains the code for the paper BIRNAT: Bidirectional Recurrent Neural Networks with Adversarial Training for Video Snapshot Compressive Imaging (The European Conference on Computer Vision 2020) by Ziheng Cheng, Ruiying Lu, Zhengjue Wang, Hao Zhang, Bo Chen, Ziyi Meng and Xin Yuan.
PyTorch > 1.3.0
numpy
scipy
The training data for BIRNAT is generated from DAVIS2017 with random crop and data argumentation and final obtain 26000 data pairs. If you want to use the same training data as ours, please run training_data / data_generation.m in MATLAB (for simulated data, we use 480p resolution; for real data, we use 1080p resolution).
The simulation test data includes six simulation data in the simulation_test folder. Three real data reconstructions for BIRNAT are in the result/real folder.
Run model without adversarial training:
python train.py
Run model with adversarial training:
python train_at.py
The adversarial training and discriminator reference this. Note that running model without adversarial training requires more than 27GB of memory and with adversarial training need 32GB which batch size is 3. Please make sure your GPU is available.
Run
python test.py
where will evaluate the preformance on simulation data using the pre-trained model in model/.
@inproceedings{Cheng20ECCV_BIRNAT,
author = {Cheng, Ziheng and Lu, Ruiying and Wang, Zhengjue and Zhang, Hao and Chen, Bo and Meng, Ziyi and Yuan, Xin},
title = {{BIRNAT}: Bidirectional Recurrent Neural Networks with Adversarial Training for Video Snapshot Compressive Imaging},
booktitle = {European Conference on Computer Vision (ECCV)},
month = {August},
year = {2020}
}