To teach others and myself the basics of deep learning I have compiled and created some resources to assist anyone looking to learn more about deep learning and all of its quirks.
The repo will consist of sparsely organized materials others and I have developed but below is my reccomended order of reading in order to get the most out of the material.
In any machine learning algorithm it is important to know your basic probability and linear algebra and this is true of deep learning as well. To get you caught up with the basics below are some resources and notebooks that I and others have used with great success
I have put together a notebook of notes that gives a quick introduction to neural networks. In those notebooks there are also references to other resources that will be good for ones first dive into neural networks and deep learning.
A good resource for a quick introduction would be Geoffry Hintons online course on neural networks. In particular lectures 1-6 are excelent introductory material.
Probably the hardest part of deep learning to understand but also the most important. If you ever wish to add your own layer types, work on improving efficiency of networks, or do pretty much anything with deep networks it is important to understand the training process which is called backpropogation. I highly reccomend reading through all of the material below several times to ensure you know the material well. Replicating any of the work below would also be a good exercise to test your knowledge.
Neural Networks and Deep Learning
Now that we are familiar with neural networks we can extend them into deep networks and explore their applications. I have provided an introductory Ipython Notebook to introduce a simple application of deep learning but more resources on why extending a neural network with deeper connections of layers gives favorable performance are given below as well.
To go deep or wide in learning?
Going Deeper with Convolutions
Very Deep Convolutional Networks for Large Scale Image Recognition
As you might know already, there are a lot of different layers one can add to a deep network besides simple fully connected layers. The two most popular in the literature these days are convolutional layers and recurrent layers. Both of these are important as they add new abilities for deep networks to learn that simple fully connected layers cannot provide. Below are some important resources to read through to understand these different layer types.
Deep learning is a booming field and has many applications. It would be impossible for me to cover them all so here is a list of applications and some important literature related to each of them.
ImageNet Classification with Deep Convolutional Neural Networks
Sequential Deep Learning for Human Action Recognition
Learning Spatiotemporal Features with 3D Convolutional Networks
Deep Neural Networks for Acoustic Modeling in Speech Recognition
Speech Recognition with Deep Recurrent Neural Networks
DeepFace: Closing the Gap to Human-Level Performance in Face Verification