oldgil / fast-artistic-videos

(Torch implementation) Fast artistic style transfer for videos, based on feedforward neural network

#Fast Artistic Style Transfer for Video
Torch implementation of a faster video stylization approach. See details and demos in project page.

Watch demo videos on YouTube: Demo Video, Comparison With Other Methods

How to Install

#install torch
cd consistencyChecker
make
cd ..
bash download_models.sh

Example for Running

Below is an example to stylize video example.mp4 (Result resolution 640:480)

• Put your video example.mp4 under this folder.
• bash fast_stylize.sh *example.mp4* *models/starry_night.t7*
• bash opt_flow.sh *example_01/frame_%06d.ppm* *example/flow_640:480*
• bash make_video.sh *example.mp4*

*___* are parts you should change accordingly.
Please refer to this project if you want to train your own style model.

Deepmatching and deepflow is super slow. Run them parallely on CPU cluster or run their GPU versions.
You just need optical flow to keep temporal consistency, not necessary deepflow. But we only tested with deepflow.

• To run parallely on k machines:
  • bash opt_flow.sh *example_01/frame_%06d.ppm* *example/flow_640:480* 1 k
  • bash opt_flow.sh *example_01/frame_%06d.ppm* *example/flow_640:480* 2 k
  • bash opt_flow.sh *example_01/frame_%06d.ppm* *example/flow_640:480* 3 k
  • ...
  • bash opt_flow.sh *example_01/frame_%06d.ppm* *example/flow_640:480* k-1 k
  • bash opt_flow.sh *example_01/frame_%06d.ppm* *example/flow_640:480* k k

Why Faster

Use the real-time algorithm for stylization of each frame. So L-BFGS is only used for temporal consistency purpose. L-BFGS takes temporal loss, perceptual loss and relation loss into account and in addition, pixel loss to avoid contrast loss.

Reference

Implementation is based on the following projects:

• Perceptual Losses for Real-Time Style Transfer and Super-Resolution
• Artistic style transfer for videos
• DeepFlow and DeepMatching

Their license can be found in their project repository.

License

GNU GPL 3.0 for personal or research use. COMMERCIAL USE PROHIBITED.