This repository contains our work on movie to cartoon translation with AnimeGANplus.
Please note that you may setup the required dependencies for running the Jupyter notebooks using
conda env create -f environment.yml
.
The role of the various files and folders is as follows:
animeganplus.ipynb
- before making any changes to the original AnimeGAN code, we carefully reorganised it into a Jupyter notebook- The
FID_tensorflow.ipynb
can be used to calculate the Frechet Inception Distance (FID) between true and generated cartoonised images. Demo input samples are in the folders: fid_samples/FID_generated_images and fid_samples/FID_original_images - The Linux commands in
ffmpeg.txt
can be used for video editing techniques such as stitching videos together downsampling, and addition of sound - The
video_maker.ipynb
contains the code for stitching output frames from our model into a final movie result - The
samples
folder containsdemo
inputs and outputs for individual frames generated using Paprika style. These frames were taken from the original video, as well as cartoonised version that we made, using the Katna keyframe extraction - The
fid_samples
folder contains two collections of sample images which can be passed toFID_tensorflow.ipynb
to compute FID - The
keyframe_extraction.ipynb
contains the code making use of Katna to perform keyframe extraction based on sum of absolute differences in LUV colorspace. The demo input sample for this code is the MBZUAI promotional video found in here. The demo output sample images are in samples/inputs folder
The three training datasets for AnimeGANplus can be downloaded using a single block of code included in the animeganplus.ipynb
file.
🎥 Sample cartoonised videos can be found in the links below:
- Animated message from Obama link
- Elon Musk cartoonised link
- Star Wars Animations link
- MBZUAI Students Visit Dubai link
Here are some pictures from out AnimeGANplus paper:
Comparison of authors (1st column) using Hayao (2nd column), Paprika (3rd column) and Shinkai styles (4th column)
Layer norm (2nd row) indeed converges faster with fewer artefacts than instance norm (1st row)
Famous actors and superheroes they play (1st row) cartoonised with AnimeGANplus in Paprika style (2nd row) and Shinkai style (3rd row)