SaulZhang / Awesome_Video_Frame_Prediction

This is a zoo which contain all kinds of video prediction model code(reimplementation or author provide)and the paper.I created this repository to make it easy for newbies in the video prediction field to get started.

Video_Prediction_ZOO

This is a zoo that contains all kinds of video frame prediction papers and code (reimplementation or offofficial implementation). The purpose of this repository is to make it easy for newbies in the video prediction field to get started.

Latest papers

Disentangling Physical Dynamics from Unknown Factors for Unsupervised Video Prediction.（CVPR,2020）

code: https://github.com/vincent-leguen/PhyDNet （empty repository）

Exploring Spatial-Temporal Multi-Frequency Analysis for High-Fidelity and Temporal-Consistency Video Prediction.（CVPR,2020）

code: None

Future Video Synthesis with Object Motion Prediction.（CVPR,2020）

code: None

Efficient and Information-Preserving Future Prediction and Beyond（ICLR,2020）

code: https://github.com/GilgameshYu/CrevNet-Traffic4cast

Improved Conditional VRNNs for Video Prediction.（ICCV,2019）

code:

Disentangling Propagation and Generation for Video Prediction.（ICCV,2019）

code:

Compositional Video Prediction.（ICCV,2019）

code: https://github.com/JudyYe/CVP （official implementation in Pytorch）

SME-Net Sparse Motion Estimation for Parametric Video Prediction through Reinforcement Learning（ICCV,2019）

code: https://github.com/hectorho0409/SME_release

Predicting_Future_Frames_Using_Retrospective_Cycle_GAN.（CVPR,2019）

Notes:https://zhuanlan.zhihu.com/p/88720676
code: https://github.com/SaulZhang/Video_Prediction_ZOO/tree/master/RetrospectiveCycleGAN （Re-Implementation）

VideoFlow: A Conditional Flow-Based Model for Stochastic Video Generation.（ICLR,2019）

code:

SDCNet: Video Prediction Using Spatially-Displaced Convolution.（ECCV,2018）

code: https://github.com/NVIDIA/semantic-segmentation/tree/master/sdcnet

ContextVP: Fully Context-Aware Video Prediction.（CVPR,2018）

code: None

Video-to-Video Synthesis.(NIPS, 2018)

code: https://github.com/NVIDIA/vid2vid

Dual Motion GAN for Future-Flow Embedded Video Prediction.（ICCV,2017）

Notes:https://zhuanlan.zhihu.com/p/100217899
code: None

Deep predictive coding networks for video prediction and unsupervised learning.（ICLR,2017）

Notes:https://zhuanlan.zhihu.com/p/99590222
code: https://github.com/coxlab/prednet （official implementation in TensorFlow）

Decomposing Motion and Content For Natural Video Sequence Prediction.（ICLR,2017）

code: https://github.com/rubenvillegas/iclr2017mcnet （official implementation in TensorFlow）

Spatio-Temporal Video Autoencoder with Differentiable Memory.(ICLR,2016)

Notes:https://zhuanlan.zhihu.com/p/106690353
code:

Deep Multi-Scale Video Prediction Beyond Mean Square Error.(ICLR,2016)

Notes:https://zhuanlan.zhihu.com/p/99104462
code: https://github.com/dyelax/Adversarial_Video_Generation （official implementation in TensorFlow）

Unsupervised Learning for Physical Interaction through Video Prediction.(NIPS,2016)

code:https://github.com/Xiaohui9607/physical_interaction_video_prediction_pytorch （unofficial implementation in TensorFlow）

Unsupervised learning of video representations using lstms.（CoRR,2015）

Notes:https://zhuanlan.zhihu.com/p/99399503
code:

Evaluation Metric

LPIPS：The Unreasonable Effectiveness of Deep Features as a Perceptual Metric.

SSIM:Image quality assessment: from error visibility to structural similarity.

Attain the relvant Dataset can refer to below website:

Caltech Pedestrian dataset：https://zhuanlan.zhihu.com/p/67506829
Kitti: https://github.com/coxlab/prednet