Style-based Drum Synthesis with GAN Inversion Demo
TensorFlow implementation of a style-based generative adversarial network for conditional synthesis of drum sounds.
Audio examples
Music created using generated drum sounds
Transforming input audio
Code
Dependencies
Python
Code has been developed with Python 3.6.13
. It should work with other versions of Python 3
, but has not been tested. Moreover, we rely on several third-party libraries, listed in requirements.txt
. They can be installed with
$ pip install -r requirements.txt
Checkpoints
The tensorflow checkpoints for loading pre-trained network weights can be download here. Unzip the folder and save it into this projects directory: "style-drumsynth/checkpoints".
Usage
Synthesis
The code for generation is contained within the drumsynth_demo.py
script, which enables conditional synthesises of drum sounds using a pretrained generator.
The following control parameters are available:
- Condition: which type of drum to generate? (kick, snare or hat)
- Direction: "synthesis controls", which principal direction to move in? [0:4]
- Direction slider: how far to move in a particular direction?
- Number of generations: how many drums to generate?
- Stocastic Variation: amount of inconsequential noise to inject into the generator layers
- Randomize: generate by randomly sampling the latent space, or generate from a fixed, pre-computed latent vectors for a kick, snare and hat
- Encode: regenerate drum sounds stored in the style-drumsynth/input_audio
- Interpolation: demonstrates waveform interpolation between two drum sounds
Generations are saved in the style-drumsynth/generations folder. To experiment encoding your own drums sounds, save your audio files in the "style-drumsynth/input_audio" directory.
drumsynth_demo.py arguments
-c CONDITION, --condition CONDITION
0: kick, 1: snare, 2:hat
-d DIRECTION, --direction DIRECTION
synthesis controls [0:4]
-ds DIRECTION_SLIDER, --direction_slider DIRECTION_SLIDER
how much to move in a particular direction
-n NUM_GENERATIONS, --num_generations NUM_GENERATIONS
number of examples to generate
-v STOCASTIC_VARIATION, --stocastic_variation STOCASTIC_VARIATION
amount of inconsequential noise injected
-r RANDOMIZE, --randomize RANDOMIZE
if set to False, a fixed latent vector is used to generate a drum sound from each condition
-e ENCODE, --encode ENCODE
regenerates drum sounds from encoder folder (True/False)
-i INTERPOLATION --interpolation INTERPOLATION
creates linear path between two random latent vectors, generates drum sounds whilst moving along the path in steps
Training
The code for training is contained within the train.py
script, which enables GAN training on a user defined dataset.
train.py main arguments
-d DATASET, --dataset DATASET
path to training data
-p PREPROC, --preproc PREPROC
set to True to prepare data for training. Ensure that drum sounds are split into class sub-folders (e.g. kicks, snares, cymbals)
-cl N_CLASSES, --n_classes N_CLASSES
-sr SAMPLE_RATE, --sr SAMPLE_RATE
-z Z_DIM --z_dim Z_DIM
-b BATCH_SIZE --batch_size BATCH_SIZE
-e EPOCHS --epochs EPOCHS
Supporting webpage
For more information, please visit the corresponding supporting website.
It contains the following:
- Audio examples
- Training data
- Generations
- Example usage within loop-based electronic music compositions
- Generating Drum Loops
- Interpolation demonstration
- Supplementary figures
- A link to the ISMIR LBD paper and post
References
References
@inproceedings{drysdale2021sds,
title={Style-based Drum Synthesis with GAN Inversion},
author={Drysdale, Jake and Tomczak, Maciej and Hockman, Jason},
booktitle = {Extended Abstracts for the Late-Breaking Demo Sessions of the 22nd
International Society for Music Information Retrieval (ISMIR) Conference.},
year={2021}
}
Help
Any questions please feel free to contact me on jake.drysdale@bcu.ac.uk