rayruchira / music_video_gen

Music Video Generation using a Deep Generative Adversarial Network

Music Video Generation using GAN's

Requirements

pip install librosa==0.7.2 torch==0.4.1 torchvision pillow scipy moviepy tqdm 

Train on custom data

To train on your own music videos, you will need to create a folder of music videos and their corresponding mp3's. To do so, place music videos in a folder and do:

ffmpeg -i name_of_video.mp4 name_of_video.mp3 for each video mp4 file. I will upload a script soon that automates this, but this has to be manual for now, sorry.

Then, simply run:

cd Progressive-GAN-pytorch

python train.py --path /path/to/folder/containing/video&music

Test on your music using pretrained checkpoint

To test on your own music file using a pretrained model, simply run:

cd Progressive-GAN-pytorch
python music2video.py --input_file /path/to/music.mp3 --outname /desired/output/videoname.mp4 --checkpoint optional/path/to/your_checkpoint_generator.pt
# if you wish to use our checkpoint, do not use the --checkpoint argument

Citation

If you use this code, please consider citing:

@misc{raymusicvideogen,
      title={Swing Dance Video Generation using ProgressiveGAN}, 
      author={Arijit Ray},
      year={2020},
      url={https://github.com/arijitray1993/music_video_gen/}
}

Acknowledgements

The NVIDIA Progressive GAN code was modified from https://github.com/odegeasslbc/Progressive-GAN-pytorch