The role of this curated list is to gather scientific articles, thesis and reports that use deep learning approaches applied to music. The list is currently under construction but feel free to contribute to the missing fields and to add other resources! To do so, please refer to the How To Contribute section. The resources provided here come from my review of the state-of-the-art for my PhD Thesis for which an article is being written. There are already surveys on deep learning for music generation, speech separation and speaker identification. However, these surveys do not cover music information retrieval tasks that are included in this repository.
Table of contents
- DL4M summary
- DL4M details
- Code without articles
- Statistics and visualisations
- Advices for reviewers of dl4m articles
- How To Contribute
- Acronyms used
- Other useful related lists
- Cited by
Each entry in dl4m.bib also displays additional information:
link- HTML link to the PDF file
code- Link to the source code if available
archi- Neural network architecture
layer- Number of layers
task- The proposed tasks studied in the article
dataset- The names of the dataset used
dataaugmentation- The type of data augmentation technique used
time- The computation time
hardware- The hardware used
note- Additional notes and information
repro- Indication to what extent the experiments are reproducible
Code without articles
- Audio Classifier in Keras using Convolutional Neural Network
- Deep learning driven jazz generation using Keras & Theano
- End-to-end learning for music audio tagging at scale
- Music Genre classification on GTZAN dataset using CNNs
- Pitch Estimation of Choir Music using Deep Learning Strategies: from Solo to Unison Recordings
- Music Genre Classification with LSTMs
- CNN based Music Emotion Classification using TensorFlow
- Separating singing voice from music based on deep neural networks in Tensorflow
- Music tag classification model using CRNN
- Finding the genre of a song with Deep Learning
- Composing music using neural nets
Statistics and visualisations
- 165 papers referenced. See the details in dl4m.bib. There are more papers from 2017 than any other years combined. Number of articles per year:
- If you are applying DL to music, there are 356 other researchers in your field.
- 34 tasks investigated. See the list of tasks. Tasks pie chart:
- 53 datasets used. See the list of datasets. Datasets pie chart:
- 30 architectures used. See the list of architectures. Architectures pie chart:
- 9 frameworks used. See the list of frameworks. Frameworks pie chart:
- Only 44 articles (26%) provide their source code. Repeatability is the key to good science, so check out the list of useful resources on reproducibility for MIR and ML.
Advices for reviewers of dl4m articles
Please refer to the advice_review.md file.
How To Contribute
Contributions are welcome! Please refer to the CONTRIBUTING.md file.
How are the articles sorted?
The articles are first sorted by decreasing year (to keep up with the latest news) and then alphabetically by the main author's family name.
Why are preprint from arXiv included in the list?
I want to have exhaustive research and the latest news on DL4M. However, one should take care of the information provided in the articles currently in review. If possible you should wait for the final accepted and peer-reviewed version before citing an arXiv paper. I regularly update the arXiv links to the corresponding published papers when available.
How much can I trust the results published in an article?
The list provided here does not guarantee the quality of the articles. You should either try to reproduce the experiments described or submit a request to ReScience. Use one article's conclusion at your own risks.
A list of useful acronyms used in deep learning and music is stored in acronyms.md.
The list of conferences, journals and aggregators used to gather the proposed materials is stored in sources.md.
- Yann Bayle (GitHub) - Instigator and principal maintainer
- Vincent Lostanlen (GitHub)
- Keunwoo Choi (GitHub)
- Bob L. Sturm (GitHub)
- Stefan Balke (GitHub)
- Jordi Pons (GitHub)
- Mirza Zulfan (GitHub) for the logo
- Devin Walters
Other useful related lists and resources
- DL4MIR tutorial with keras - Tutorial for Deep Learning on Music Information Retrieval by Thomas Lidy
- Video talk from Ron Weiss - Ron Weiss (Google) Talk on Training neural network acoustic models on waveforms
- Slides on DL4M - A personal (re)view of the state-of-the-art by Jordi Pons
- DL4MIR tutorial - Python tutorials for learning to solve MIR tasks with DL
- Awesome Python Scientific Audio - Python resources for Audio and Machine Learning
- ISMIR resources - Community maintained list
- ISMIR Google group - Daily dose of general MIR
- Awesome Python - Audio section of Python resources
- Awesome Web Audio - WebAudio packages and resources
- Awesome Music - Music softwares
- Awesome Music Production - Music creation
- The Asimov Institute - 6 deep learning tools for music generation
- DLM Google group - Deep Learning in Music group
- MIR community on Slack - Link to subscribe to the MIR community's Slack
- Unclassified list of MIR-related links - Cory McKay's list of various links on DL, MIR, ...
- MIRDL - Unmaintained list of DL articles for MIR from Jordi Pons
- WWW 2018 Challenge - Learning to Recognize Musical Genre on the FMA dataset
- Music generation with DL - List of resources on music generation with deep learning
- Auditory Scene Analysis - Book about the perceptual organization of sound by Albert Bregman, the "father of Auditory Scene Analysis".
- Demonstrations of Auditory Scene Analysis - Audio demonstrations, which illustrate examples of auditory perceptual organization.
- AudioContentAnalysis nearly exhaustive list of music-related datasets
- Teaching MIR
- Wikipedia's list of datasets for machine learning research
- Datasets for deep learning
- Awesome public datasets
- Awesome music listening
- DLPaper2Code: Auto-generation of Code from Deep Learning Research Papers -
- Model Convertors - Convertors for DL frameworks and backend
- Deep architecture genealogy - Genealogy of DL architectures
- Deep Learning as an Engineer - Slides from Jan Schlüter
- Awesome Deep Learning - General deep learning resources
- Awesome Deep Learning Resources - Papers regarding deep learning and deep reinforcement learning
- Awesome RNNs - RNNs code, theory and applications
- Cheatsheets AI - Cheat Sheets for Keras, neural networks, scikit-learn,...
- DL PaperNotes - Summaries and notes on general deep learning research papers
- General lists
- Echo State Network
- DL in NLP - Best practices for using neural networks by Sebastian Ruder
- CNN overview - Stanford Course
- Dilated Recurrent Neural Networks - How to improve RNNs?
- Encoder-Decoder in RNNs - How Does Attention Work in Encoder-Decoder Recurrent Neural Networks
- On the use of DL - Misc fun around DL
- ML from scratch - Python implementations of ML models and algorithms from scratch from Data Mining to DL
- Comparison of DL frameworks - Presentation describing the different existing frameworks for DL
- ELU > ReLU - Article describing the differences between ELU and ReLU
- Reinforcement Learning: An Introduction - Book about reinforcement learning
- Estimating Optimal Learning Rate - Blog post on the learning rate optimisation
- GitHub repo for sklearn add-on for imbalanced learning - ML in uneven datasets
- Video on DL from Nando de Freitas, Scott Reed and Oriol Vinyals - Deep Learning: Practice and Trends (NIPS 2017 Tutorial, parts I & II)
- Article "Are GANs Created Equal? A Large-Scale Study" - Actually comparing DL algorithms
- Battle of the Deep Learning frameworks - DL frameworks comparison and evolution
- Black-box optimization - There are other optimization algorithms than just gradient descent
If you use the information contained in this repository, please let us know! This repository is cited by:
- Alexander Schindler
- Meinard Müller, Christof Weiss, Stefan Balke
- WWW 2018 Challenge: Learning to Recognize Musical Genre
- Awesome Deep Learning
You are free to copy, modify, and distribute Deep Learning for Music (DL4M) with attribution under the terms of the MIT license. See the LICENSE file for details. This project use another projects and you may refer to them for appropriate license information :
- Readme checklist - To build an universal Readme.
- Pylint - To clean the python code.
- Numpy - To manage python structure.
- Matplotlib - To plot nice figures.
- Bibtexparser - To deal with the bib entries.