DougFelipe / deeplearning

Repository for IMD1114, an undergraduate course about Deep Learning

Universidade Federal do Rio Grande do Norte

Instituto Metrópole Digital

IMD1114 - Deep Learning

References

• 📚 Adrian Rosebrock. Deep Learning for Computer Vision with Python [Link]
• 📚 Jason Brownlee. Deep Learning for Computer Vision: Image Classification, Object Detection, and Face Recognition in Python [Link]
• 📚 Jason Brownlee. Better Deep Learning [Link]
• 📚 Aurélien Géron. Hands on Machine Learning with Scikit-Learn, Keras and TensorFlow. [Link]
• 📚 François Chollet. Deep Learning with Python [Link]

Lessons

Week 01 - Introduction [Slide]

• Course Outline & Presentation
• Google Colab Introduction [Video]
• Google Colab Cont. [Video]
• Functions, Context Managers and Decorators
• Object-Oriented, Functional Programming

Weeks 02 & 03 - Fundamentals of Deep Learning [Slide]

• Outline [Video]
• The perceptron [Video]
• Building Neural Networks [Video]
• Matrix Dimension [Video]
• Applying Neural Networks [Video]
• Training a Neural Networks [Video]
• Backpropagation with Pencil & Paper [Video]
• Learning rate & Batch Size [Video]
• Exponentially Weighted Average [Video]
• Adam, Momentum, RMSProp, Learning Rate Decay [Video]
• TensorFlow Crash Course
• Fundamentals of TensorFlow
• Optimization Methods
• Better Learning - Part I

Weeks 04 & 05 Better Generalization vs Better Learning [Slide]

• Better Generalization
  • Outline [Video]
  • Spliting Data [Video]
  • Bias vs Variance [Video]
  • Weight Regularization [Video]
  • Weight Constraint [Video]
  • Dropout [Video]
  • Promote Robustness with Noise [Video]
  • Early Stopping [Video]
  • Better Generalization - Part I
  • Better Generalization - Part II
• Better Learning
  • Data scaling [Video]
  • Vanishing/Exploding Gradient [Video]
  • Fix Vanishing Gradient with Relu [Video]
  • Fix Exploding Gradient with Gradient Clipping [Video]
  • Better Learning - Part II
  • Better Learning - Part III

Week 06 - Hyperparameter Tuning & Batch Normalization [Slide]

• Outline [Video]
• Hyperparameter Tuning Fundamentals [Video]
• Keras Tuner, and Weight and Biases [Video]
• Wandb - Part 01 [Video]
• Wandb - Part 02 [Video]
• Batch Normalization Fundamentals [Video]
• Batch Normalization Math Details [Video]
• Batch Normalization Case Study [Video]
• Hyperparameter tuning using keras tuner
• Hyperparameter tuning using weights and biases
• Batch Normalization

Week 07 - Fundamentals of Convolutional Neural Networks (CNN) [Slide]

• Outline [Video]
• CNN: Introduction and motivation [Video]
• Convolutional layer [Video]
• Case study of convolutional layer [Video]
• Pooling layer [Video]
• Fully connected layer [Video]
• Case study - signs dataset [Video]
• Foundations of DL for CV
• CNN case study

Week 08 - Convolutional Neural Networks (CNN) Architecture I [Slide]

• Outline [Video]
• Typical CNN Architecture [Video]
• Best practices when building your own CNN [Video]
• LeNet-5 [Video]
• Training LeNet-5 using MNIST dataset [Video]
• ImageNet & ILSVRC [Video]
• AlexNet [Video]
• Dogs vs Cats Challenge + HDF5 [Video]
• CNN Architectures - Hands on Part #01 [Video]
• CNN Architectures - Hands on Part #02 [Video]
• LeNet-5 and AlexNet applied to Cat & Dogs problem and using HDFS files

Week 09 - Convolutional Neural Networks (CNN) Architecture II [Slide]

• Outline [Video]
• VGG [Video]
• Case study using CIFAR-10 [Video]
• How to use 1x1 convolutions [Video]
• GoogLeNet [Video]
• MiniGoogleLeNet using CIFAR-10 [Video]
• DeeperGoogLeNet and Tiny ImageNet Challenge [Video]
• VGG and GoogLeNet hands on

Week 10 - Convolutional Neural Networks (CNN) Architecture III - ResNet

• ResNet hands on

Week 11 - Transfer Learning [Slide]

• Outline [Video]
• Feature extractors [Video]
• Feature extractors: case study [Video]
• Fine-tuning [Video]
• Fine-tuning: case study I - flowers 17 [Video]
• Fine-tuning: case study II - cats and dogs [Video]
• Transfer Learning hands on
• Case Study: cats and dogs