Use a DCNN to perform frame interpolation.
Paper: Deep Motion: A Convolutional Neural Network for Frame Interpolation
Based off of the U-Net architecture
- Keras (tested on v1.1.2)
- TensorFlow (tested on v0.10.0)
- NumPy, SciPy, matplotlib
- OpenCV (tested on v3.1.0, but v2.X should work) (only needed for fps_convert.py)
- FFMPEG (only needed for batch samples generator for YouTube-8M videos)
Download the model weights here.
*Note that the weights are trained using the architecture defined in FI_unet.py/get_unet_2(), which requires input of shape=(6, 128, 384), due to the use of Batch Normalization (probably could do without that)
Details in train.py. It's Keras, so don't worry ;)
For images, look at DO_TESTING section of train.py
For videos, you can use fps_convert.py to double/quadruple/etc the FPS of any video
View the results at the end of the paper
Watch the presentation video