Orchestral Music
Utayaki opened this issue · comments
Hi there!
The neural network works brilliantly,
Thanks you a lot.
But...
It only makes piano pattern
Could it also make the full orchestra music with strings, winds and percussion?
And if it could, HOW?!
I haven't tried this yet, but music21 supports different instruments https://github.com/cuthbertLab/music21/blob/a78617291ed0aeb6595c71f82c5d398ebe604ef4/music21/instrument.py
Hi,
Music21 supports most instruments that you could need so that should cover creating the music.
On the neural network end, there are a few ways you can go about generating full orchestral music.
First is MuseGAN (https://arxiv.org/abs/1709.06298) which uses a GAN to generate the music, this can be fairly tricky since GANs tend to be tough to work with.
Second, expand the network we have in this repository into a multi-input multi-output network, where each input corresponds to separate instrument and each output corresponds to a separate instrument. By using that architecture you will ensure that throughout the majority of the network, it takes into account the input from all of the instruments, while the output layers only focus on generating for a single instrument.
Third (if you're feeling really daring), you could try your hand at implementing MuseNet from OpenAI (https://openai.com/blog/musenet/). ;)
I have been working on a version of this repo that supports outputting the note/chord, duration, and offset which you can find here
The code is a bit messy right now but the TF2 version runs fine, I need to clean it up when I have some time
So long as you can create a vocabulary of musical instruments with Music21 (similarly to how I expanded it to create vocabs of the durations and offsets), then you could add a third branch which classifies the instrument and in the generator sets the instrument of the output at that step prior to compiling the midi file. I believe then you'd be able to synthesise a whole orchestra.
Although now I think about it, when parsing the file you will also have to separate each instrument as individual entities: i.e. if a C major chord is played on piano and a lower C major chord is played by a double bass at the same time, then you will have to extract them as two seperate chords with the second one being at an offset of 0 - so, then, when a generator creates something like this, it'll offset the first instrument and then have 0 offset for the second one so they effectively play at the same time