omaniasy / VirtualConductor

VirtualConductor

• 2021.4.02 First Commit
• 2021.4.20 New pre-trained models (with motion decomposition) are released. Training codes and dataset will be available soon! (about one week or more)

Testing

Place your music (.mp3 seems to usually have problems, please try .wav) file(s) in ./test/testset/ (we have prepared some music samples), then run test.py, the .mp4 results will be saved in ./test/results/

environment: pytorch==1.6.0,

needs ffmpeg, moviepy for visualize and generate mp4 video.

If you are using CPU only machine, change line 16-25 of test.py to:

if G_high_dir is not None:    
    G_high = torch.load(G_high_dir,map_location=torch.device('cpu'))#.cuda()    
    G_high.eval()    
    print('high-pass model loaded from', G_high_dir)    
    mode.append('high')
if G_low_dir is not None:    
    G_low = torch.load(G_low_dir,map_location=torch.device('cpu'))#.cuda()    
    G_low.eval()    
    print('low-pass model loaded from', G_low_dir)    
    mode.append('low')

Papers

This repo is the implementation of our paper which is under peer-review. If you are interested in our project, you can request for a pdf manuscript.