Rough list of my favorite deep learning resources, useful for revisiting topics or for reference. I (Guillaume Chevalier) have got through all of the content listed there, carefully, and built this list.
Discover by yourself now and explore why on Google Trends
Moores Law about exponential progress rates in computer science hardware is now more affecting GPUs than CPUs because of physical limits and will slowly shift toward parallel architectures [read more]. Deep learning exploits such architecture under the hood.
- Machine Learning by Andrew Ng on Coursera (Good entry-level online class with certificate. Taught by: Andrew Ng, Associate Professor, Stanford University; Chief Scientist, Baidu; Chairman and Co-founder, Coursera)
- Neural networks class - Université de Sherbrooke (Very interesting class available online for free by Hugo Larochelle)
- Deep Learning by Google (Very good class covering high-level deep learning concepts, it is the logical next step after Andrew Ng's Machine Learning class)
- Machine Learning for Trading by Georgia Tech (Interesting class for acquiring basic knowledge of machine learning applied to trading and some AI and finance concepts)
- The Unreasonable Effectiveness of Recurrent Neural Networks (MUST READ post by Andrej Karpathy - this is what motivated me to learn RNNs)
- Neural Networks, Manifolds, and Topology (Fresh look on how neurons map information)
- Understanding LSTM Networks (Walktrough the LSTM cells' inner workings, plus interesting links in conclusion)
- Attention and Augmented Recurrent Neural Networks (Interesting for visual animations)
- Recommending music on Spotify with deep learning (Awesome for doing clustering on audio - post by an intern at Spotify)
- Announcing SyntaxNet: The World’s Most Accurate Parser Goes Open Source (Parsey McParseface's birth)
- Improving Inception and Image Classification in TensorFlow (Very interesting CNN architecture)
- WaveNet: A Generative Model for Raw Audio (Realistic talking machines)
- François Chollet's Twitter (Author of Keras - has interesting Twitter posts)
- TensorFlow's GitHub repository (Most known deep learning framework, both flexible and high-level)
- skflow (TensorFlow wrapper à la scikit-learn)
- Keras (Keras is another very intersting deep learning framework like TensorFlow)
- carpedm20's repositories (Many interesting neural network architectures are implemented by the Korean guy Taehoon Kim - A.K.A. carpedm20)
- carpedm20/NTM-tensorflow (NTM TensorFlow implementation)
- Deep learning for lazybones (CNN vision transfer learning tutorial in TensorFlow from pretrained AlexNet 2012)
- LSTM for Human Activity Recognition (Tutorial of mine on using LSTMs on time series for classification)
- ML / DL repositories I starred (GitHub is full of nice code samples & projects)
- UCI Machine Learning Repository (TONS of datasets for ML)
- Cornell Movie--Dialogs Corpus (Could be used for a chatbot)
- SQuAD The Stanford Question Answering Dataset (Interesting QA dataset)
- Aligned Hansards of the 36th Parliament of Canada (Aligned text chunks useful for FR-EN machine translation)
- Neural Networks and Deep Learning (Overview)
- Yes you should understand backprop (Exposing backprop's caveats)
- Filtering signal, plotting the STFT and the Laplace transform (Simple demo of mine on signal processing)
- Window Functions (Interesting Wikipedia page listing known window functions)
- MathBox, Tools for Thought Graphical Algebra and Fourier Analysis (New look on Fourier analysis)
- How to Fold a Julia Fractal (Very interesting animations dealing with complex numbers and wave equations)
- Animate Your Way to Glory, Math and Physics in Motion (Look on convergence methods in physic engines and interaction design)
- Animate Your Way to Glory - Part II, Math and Physics in Motion (Nice animations for rotation and rotation interpolation with Quaternions)
- Deep Learning in Neural Networks: An Overview (You_Again's DL summary, mostly about RNNs)
- Bidirectional Recurrent Neural Networks (Better classifications with RNNs!)
- Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation (Two networks in one combined into a seq2seq (sequence to sequence) Encoder–Decoder architecture. RNN Encoder–Decoder with 1000 hidden units. Adadelta optimizer.)
- Sequence to Sequence Learning with Neural Networks (4 stacked LSTM cells of 1000 hidden size with reversed input sentences, and with beam search, on the WMT’14 English to French dataset)
- Exploring the Limits of Language Modeling (Nice recursive models using word level LSTMs on character level CNN using an overkill amount of GPU power)
- Exploring the Depths of Recurrent Neural Networks with Stochastic Residual Learning (Basically, residual connections can be better than stacked RNNs in the presented case of sentiment analysis)
- Neural Turing Machines (Outstanding for implementing simple neural algorithms with seemingly good generalisation)
- Teaching Machines to Read and Comprehend (A very interesting and creative work about textual question answering, there is something to do with that)
- Pixel Recurrent Neural Networks (Nice for photoshop-like "content aware fill" to fill missing patches in images)
- Adaptive Computation Time for Recurrent Neural Networks (Very interesting, I would love to see how well would it combines to Neural Turing Machines. Interesting interactive visualizations on the subject can be found here.)
- Hybrid computing using a neural network with dynamic external memory (Improvements on differientable memory based on NTM: now differentiable neural computer)
- What is the Best Multi-Stage Architecture for Object Recognition? (For the use of "local contrast normalization")
- ImageNet Classification with Deep Convolutional Neural Networks (AlexNet, 2012 ILSVRC, breakthrough of the ReLU activation function)
- Visualizing and Understanding Convolutional Networks (For the "deconvnet layer")
- Very Deep Convolutional Networks for Large-Scale Image Recognition (For the idea of stacking multiple 3x3 conv+ReLU before pooling for a bigger filter size with few parameters, also there is a nice table for "ConvNet Configuration")
- Going Deeper with Convolutions (GoogLeNet: Appearance of "Inception" layers/modules, the idea is of parallelizing conv layers into many mini-conv of different size with "same" padding, concatenated on depth)
- Highway Networks (Highway networks: residual connections)
- Batch Normalization: Accelerating Deep Network Training b y Reducing Internal Covariate Shift (Batch normalization (BN): to normalize a layer's output by also summing over the entire batch, and then performing a linear rescaling and shifting of a certain trainable amount)
- Deep Residual Learning for Image Recognition (Very deep residual layers with batch normalization layers - a.k.a. "how to overfit any vision dataset with too many layers")
- Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning (For improving GoogLeNet with residual connections)
- WaveNet: a Generative Model for Raw Audio (Epic raw voice/music generation with new architectures based on dilated causal convolutions to capture more audio lenght)
- Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling (3D-GAN for 3D model generation and fun 3D furniture arithmetics from embeddings)
- Deep Learning & Machine Learning (Advanced topics) (Awesome videos)
- Tensor Calculus and the Calculus of Moving Surfaces (Generalize properly how Tensors work, yet watching a few videos helps)
- Signal Processing YouTube Playlist (A YouTube playlist I composed about DFT/FFT, STFT and the Laplace transform - I was mad about my software engineering bachelor not including signal processing classes)
- Computer Science (Yet another YouTube playlist I composed, this time about various CS topics just slightly related)
- Siraj's YouTube Channel (Siraj has some entertaining videos)
- Two Minute Papers' Youtube Channel (Interesting but very shallow overview of some papers, such as WaveNet)
- Quora.com (Finest question/answer site)
- Hacker News (Maybe how I discovered ML - Interesting trends appear on that site way before they get to be a big deal)
- DataTau (A hub similar to Hacker News but specific to data science)
- Naver (Korean search engine - best used with Google Translate, ironically. Surprisingly, sometimes deep learning results and comprehensible advanced math content shows up more easily there than on Google search)
- Arxiv Sanity Preserver (arXiv browser with TF/IDF features)