Erfaan-Rostami / Deep-Learning

Useful Information About Deep Learning

Deep-Learning

✷ Useful information and links about Deep Learning

• you can find a good course that provides a thorough understanding of the fundamental concepts and recent advances in deep learning here by Aykut Erdem .

• Practical courses(useful and attractive!) about Deep Learning

• Deep Learning Garden: Liping's machine learning, computer vision, and deep learning home: resources about basics, applications, and many more…

• http://deeplearning.cs.cmu.edu/

• the page "Why Momentum Really Works" is a popular story about momentum by Gabriel Goh .

• What does end to end mean in deep learning methods?

• Zero to Deep Learning; San Francisco’s leading machine & deep learning bootcamp.

• 30 Amazing Applications of Deep Learning, by YARON HADAD

• The 10 Deep Learning Methods AI Practitioners Need to Apply, by James Le

Optimization

• Useful links and papers:

  ∘ Batch Normalization in Deep Networks by Sunita Nayak

  ∘ Epoch vs Batch Size vs Iterations by SAGAR SHARMA

Convolutional Networks

• Useful links and papers:

  ∘ Depth wise Separable Convolutional Neural Networks

  ∘ A Basic Introduction to Separable Convolutions by Chi Feng Wang

  ∘ Convolutional Neural Networks (CNNs / ConvNets)

RECURRENT AND RECURSIVE NEURAL NETWORKS

• Useful links and papers:

  ∘ Reservoir computing

  ∘ Gentle introduction to Echo State Networks

  ∘ Animated RNN, LSTM and GRU

  ∘ What is the realitionship between deep learning methods and reservoir computing (if any)?

Autoencoder

• Useful links and papers:

  ∘ Sparse Autoencoder; Lecture note by Andrew Ng

  ∘ k-Sparse Autoencoders

  ∘ Tutorial - What is a variational autoencoder?by JAAN ALTOSAAR

  ∘ A good PowerPoint by Shilin HE

Main TextBooks

♥ "Better Deep Learning"; a new eBook written by Jason Brownlee in the friendly Machine Learning Mastery style that you’re used to, discover exactly how to improve the performance of deep learning neural network models on your predictive modeling projects.

Google Colab and Anaconda

• Google Colab provides GPU and it’s totally free. Seriously!

• A Simple Tutorial for the Frustrated and Confused, by Anne Bonner

• Google Colab - The Beginner’s Guide, by Vishakha Lall

• Managing environments

• Conda commands(create virtual environments for python with conda)

About Keras

⇝ Keras is a Deep Learning library for Python, that is simple, modular, and extensible.

✦ How to Configure Image Data Augmentation in Keras; by Jason Brownlee

✦ Keras ImageDataGenerator and Data Augmentation; by Adrian Rosebrock

✦ Building powerful image classification models using very little data

✦ Everything you need to know about Keras to build your first deep learning model, by Pallawi

About PyTorch

⇝ As described in PyTorch page, it's An open source machine learning framework that accelerates the path from research prototyping to production deployment.

⇝ generating data in parallel with PyTorch

Regularization

Many strategiesused in machine learning are explicitly designed to reduce the test error, possiblyat the expense of increased training error. These strategies are known collectivelyas regularization.

• Chapter 7 of the Deep Learning textbook.

• An Overview of Regularization Techniques in Deep Learning (with Python code), by SHUBHAM JAIN.

• How to Improve a Neural Network With Regularization(with code for L2 regularization and dropout), by Marco Peixeiro

Neural Activation Functions

⚑ Understanding Activation Functions: Beginners guide to Activation Functions

⚑ Learning Activation Functions to Improve Deep Neural Networks

⚑ Mish: A Self Regularized Non-Monotonic Neural Activation Function, by Diganta Misra

⚑ 7 Types of Neural Network Activation Functions: How to Choose?

Generative Adversarial Networks(GANs)

✸ Generative Adversarial Nets(GANs), a new framework for estimating generative models via an adversarial process.

✸ GANs from Zero to Hero: Best Resources for Newcomers; by Oleksii Kharkovyna

✸ Understanding GANs: Building, step by step, the reasoning that leads to GANs; by Joseph Rocca

Challenges and Competitions in AI

• Grand Challenges in Biomedical Image Analysis
• Kaggle

Boltzmann Machines

• Useful links and papers:

  ♣ Boltzmann Machines; Geoffrey E. Hinton

• Slides:

  ♣ Deep belief nets; by Marcus Frean

Skip Connections

Variational Autoencoders(VAE)

• Useful links and papers:

  ☑ Tutorial on Variational Autoencoders

  ☑ Understanding Variational Autoencoders; by Joseph Rocca

• Slides:

  ☑ Variational Autoencoders (VAEs); by Yuqin Yang

  Deep Learning Frameworks

• MXNet

• PyTorch

• TensorFlow

• Theano

• Torch