Deep Learning Nanodegree Foundation Notes & Exercises
Notes and coding exercises from the various lessons in Udacity's Deep Learning Nanodegree Foundation
Lessons
1 - Intro to Neural Networks
- PROJECT 1: Bike Sharing Neural Network (Repo)
2 - Sentiment Analysis
Sentiment Analysis with Andrew Trask
- Framing Problems for Neural Networks (Notebook)
- Creating the Input/Output Data (Notebook)
- Building the Neural Network (Notebook)
- Making Learning Faster by Reducing Noise (Notebook)
- Making our Network Train and Run Faster (Notebook)
- Reducing Noise by Strategically Reducing Noise (Notebook)
Intro to TFLearn
- Sentiment Analysis with TFLearn (Notebook)
- Handwritten Digit Recognition with TFLearn and MNIST (Notebook)
3 - MiniFlow
- Forward Propagation (Notebook)
- Learning and Loss (Notebook)
- Linear Transform (Notebook)
- Sigmoid Function (Notebook)
- Cost (Notebook)
- Gradient Descent (Notebook)
4 - Intro to TensorFlow
5 - Deep Neural Networks
6 - Convolutional Neural Networks
-
PROJECT 2: Image Classification Convolutional Neural Network (Repo)
7 - Recurrent Neural Networks
-
Recurrent Neural Network Exercise (Anna Karenina Text Generation) (Notebook)
-
Sentiment Analysis Recurrent Neural Network (Notebook)
-
PROJECT 3: Simpsons Script Generation with Recurrent Neural Networks (Repo)
Resources
- Lecture on RNNs and LSTM by Andrej Karpathy (CS231n - Stanford)
- Understanding LTSM Networks by Christopher Olah (Blog)
- The Unreasonable Effectiveness of Recurrent Neural Networks by Andrej Karpathy (Blog)
- A Beginner's Guide to Recurrent Networks and LSTMs (Blog)
- Recurrent Neural Networks in TensorFlow (Tutorial)
- RNNs in TensorFlow (Blog)
8 - Embeddings and Word2Vec
- Embeddings and Word2Vec (Skip-Gram) Exercise (Notebook)
Resources
- Word2Vec Overview: The Skip-Gram Model by Chris McCormick (Blog)
- Efficient Estimation of Word Respresentations in Vector Space by Mikolov et al. (Paper) [PDF] - (original Word2Vec Paper)
- Distributed Representations of Words and Phrases and their Compositionality by Mikolov et al. (Paper) [PDF] - (improvements to orignial paper)
- Word2Vec: NLP with Deep Learning with TensorFlow by Thushan Ganegedara (Blog)
- TensorFlow Word2Vec Tutorial (Tutorial)
9 - TensorBoard
- Viewing Graphs (Notebook)
- Name Scopes (Notebook)
- Inspecting Variables - TF Summaries (Notebook)
- Choosing Hyperparameters (Notebook)
Resources
- Hands-on TensorBoard Tutorial (TensorFlow Dev Summit 2017)
- TensorBoard: Visualizing Learning (Tutorial)
10 - Weight Initialization
- Comparing Different Weight Initializations on MNIST (Notebook)
Resources
- Understanding the Difficulty of Training Deep Feedforward Neural Networks by Bengio et al. (Paper) [PDF]
- Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification by Microsoft Research (Paper) [PDF]
- Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift by Google Research (Paper) [PDF]
11 - Transfer Learning
- Transfer Learning with VGGNet (Notebook)
12 - Sequence to Sequence RNN
-
Character Sequence to Sequence (Notebook)
-
PROJECT 4: English to French Translation with Sequence to Sequence RNN (Repo)
13 - Reinforcement Learning
- Playing Cart-Pole using Q-Learning (Notebook)