Awesome-VFI
This is a summary of recent video frame interpolation (VFI) methods.
| Title | Model | Published | Repo | Keywords |
|---|---|---|---|---|
| Video Frame Interpolation via Adaptive Convolution | -- | CVPR2017 | -- | Kernel-based |
| Video Frame Interpolation via Adaptive Separable Convolution | SepConv | ICCV2017 | Pytorch&Cupy  |
Kernel-based |
| Video Frame Synthesis using Deep Voxel Flow | DVF | ICCV2017 | TensorFlow  |
Voxel flow |
| Context-aware Synthesis for Video Frame Interpolation | CtxSyn | CVPR2018 | -- | Flow-based, Content |
| Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolation | Super SloMo | CVPR2018 | Pytorch  |
Flow-based, Arbitrary time |
| Video Enhancement with Task-Oriented Flow | ToFlow | IJCV2019 | Torch&Lua  |
Flow-based, Vimeo-90K |
| Depth-Aware Video Frame Interpolation | DAIN | CVPR2019 | Pytorch  |
Flow-based, Depth-guided |
| PhaseNet for Video Frame Interpolation | PhaseNet | CVPR2019 | -- | Phase-based |
| IM-Net for High Resolution Video Frame Interpolation | IM-Net | CVPR2019 | -- | Kernel-based |
| Unsupervised Video Interpolation Using Cycle Consistency | -- | ICCV2019 | Pytorch  |
Unsupervised, Cycle consistency |
| Deep Video Frame Interpolation Using Cyclic Frame Generation | CyclicGen | AAAI2019 | TensorFlow  |
Cycle consistency |
| Quadratic Video Interpolation | QVI | NIPS2019 | Pytorch&Cupy  |
Multiple frames |
| AdaCoF: Adaptive Collaboration of Flows for Video Frame Interpolation | AdaCoF | CVPR2020 | Pytorch  |
Kernel-based, Deformable convolution |
| Softmax Splatting for Video Frame Interpolation | SoftSplat | CVPR2020 | Pytorch&Cupy  |
Flow-based, Forward warping |
| FeatureFlow: Robust Video Interpolation via Structure-to-texture Generation | FeFlow | CVPR2020 | Pytorch  |
Flow-based, Kernel-based, Deformable convolution |
| BMBC: Bilateral Motion Estimation with Bilateral Cost Volume for Video Interpolation | BMBC | ECCV2020 | Pytorch  |
Flow-based, Cost volume |
| Channel Attention Is All You Need for Video Frame Interpolation | CAIN | AAAI2020 | Pytorch  |
Channel attention |
| FLAVR: Flow-Agnostic Video Representations for Fast Frame Interpolation | FLAVR | Arxiv.2012 | Pytorch  |
3d convolution |
| CDFI: Compression-Driven Network Design for Frame Interpolation | CDFI | CVPR2021 | Pytorch  |
Model compression |
| Deep Animation Video Interpolation in the Wild | AnimeInterp | CVPR2021 | Pytorch  |
Amine interpolation, ATD-12K |
| XVFI: eXtreme Video Frame Interpolation | XVFI | ICCV2021 | Pytorch  |
Flow-based, X4K1000FPS |
| Asymmetric Bilateral Motion Estimation for Video Frame Interpolation | ABME | ICCV2021 | Pytorch  |
Flow-based, Multiple flows, Cost volume |
| Revisiting Adaptive Convolutions for Video Frame Interpolation | -- | WACV2021 | Pytorch&Cupy  |
Kernel-based |
| IFRNet: Intermediate Feature Refine Network for Efficient Frame Interpolation | IFRNet | CVPR2022 | Pytorch  |
Flow-based, Efficiency |
| FILM: Frame Interpolation for Large Motion | FILM | CVPR2022 | Pytorch  |
Flow-based, Large motion |
| Video Frame Interpolation Transformer | VFIT | CVPR2022 | Pytorch  |
Window-based Attention, Multiple Frames |
| Video Frame Interpolation with Transformer | VFIFormer | CVPR2022 | Pytorch  |
Flow-based, Window-attention |
| Long-term Video Frame Interpolation via Feature Propagation | P-INet | CVPR2022 | -- | Flow-based |
| ST-MFNet: A Spatio-Temporal Multi-Flow Network for Frame Interpolation | ST-MFNet | CVPR2022 | Pytorch  |
Flow-based, Multiple frames |
| Many-to-many Splatting for Efficient Video Frame Interpolation | M2M-VFI | CVPR2022 | Pytorch&Cupy  |
Flow-based, Forward warping, Multiple flows |
| Real-Time Intermediate Flow Estimation for Video Frame Interpolation | RIFE | ECCV2022 | Pytorch  |
Flow-based, Distillation |
| Improving the Perceptual Quality of 2D Animation Interpolation | -- | ECCV2022 | Pytorch  |
Animation Interpolation, New metrics |
| Cross-Attention Transformer for Video Interpolation | TAIN | Arxiv.2207 | -- | Cross-Attention, Transformer |
| Error-Aware Spatial Ensembles for Video Frame Interpolation | -- | Arxiv.2207 | -- | Flow-based, Divide-and-conquer |
| TTVFI: Learning Trajectory-Aware Transformer for Video Frame Interpolation | -- | Arxiv.2207 | -- | Flow-based, Transformer |
| Splatting-based Synthesis for Video Frame Interpolation | -- | WACV2023 | -- | Flow-based, Forward warp |
| A Unified Pyramid Recurrent Network for Video Frame Interpolation | -- | CVPR2023 | -- | Flow-based, Forward warping, Cost volume |
| Exploring Motion Ambiguity and Alignment for High-Quality Video Frame Interpolation | -- | CVPR2023 | -- | -- |
| Frame Interpolation Transformer and Uncertainty Guidance | -- | CVPR2023 | -- | -- |
| Exploring Discontinuity for Video Frame Interpolation | -- | CVPR2023 | -- | Discontinuious motion |
| BiFormer: Learning Bilateral Motion Estimation via Bilateral Transformer for 4K Video Frame Interpolation | BiFormer | CVPR2023 | TBD  |
Flow-based, Transformer, Cost Volume |
| Extracting Motion and Appearance via Inter-Frame Attention for Efficient Video Frame Interpolation | EMA-VFI | CVPR2023 | Pytorch  |
Flow-based, Transformer |
| Range-nullspace Video Frame Interpolation with Focalized Motion Estimation | -- | CVPR2023 | -- | -- |
| AMT: All-Pairs Multi-Field Transforms for Efficient Frame Interpolation | AMT | CVPR2023 | Pytorch | Flow-based, All-pair Cost volume, Multiple flows |
*The keywords Multiple Frames means the number of input frames is more than two.
*Above methods exclude Spatial-Temporal Super Resolution, Event-based Frame Interpolation and other joint tasks (e.g. deblur+VFI). Coming soon! (Perhaps not soon)