- I emphasize mathematical/conceptual foundations because implementations of ideas(ex. Torch, Tensorflow) will keep evolving but the underlying backbone must be sound. Anybody with an interest in deep learning can and should try to understand why things work.
- I include neuroscience as a useful conceptual foundation for two reasons. First, it may provide inspiration for future models and algorithms. Second, the success of deep learning can contribute to useful hypotheses and models for computational neuroscience.
- Why does unsupervised pre-training help deep learning?(Erhan et al. 2010)
- Learning Deep Generative Models(Salakhutdinov 2015)
- Dropout as a Bayesian Approximation(Yarin Gal 2016)
- Dropout Rademacher Complexity of Deep Neural Networks(Wei Gao 2014)
- Markov Chain Monte Carlo and Variational Inference: Bridging the Gap (Salimans 2014)
- Opening the black box of Deep Neural Networks via Information (Schwartz-Ziv 2017)
- Uncertainty in Deep Learning(Yarin Gal 2017)
- Distribution-Specific Hardness of Learning Neural Networks(Shamir 2017)
- Lessons from the Rademacher Complexity for Deep Learning(Sokolic 2016)
- Shannon Information and Kolmogorov Complexity (Grunwald 2010)
- A mathematical theory of Deep Convolutional Neural Networks for Feature Extraction(Wiatowski 2016)
- Spectral Representations for Convolutional Neural Networks(Rippl 2015)
- Electron-Proton dynamics in deep learning(Zhang 2017)
- Towards an integration of deep learning and neuroscience(Marblestone 2016)
- Equilibrium Propagation(Scellier 2016)
- Biologically plausible deep learning(Bengio 2015)
- Random synaptic feedback weights(Lillicrap 2016)
- Deep learning with spiking neurons(Mesnard 2016)
- Towards deep learning with spiking dendrites(Guergiuev 2017)