JiahongChen / d2l-pytorch-implementation

This repo provides Pytorch implementation for codes in the book "Dive Into Deep Learning" (http://d2l.ai/) and course Berkeley STAT 157 (https://courses.d2l.ai), which gives a brief tutorial on deep learning methods.

Pytorch Implementation for book "Dive Into Deep Learning" and course Berkeley STAT 157

This repo reproduces codes in the course STAT 157 (UC Berkeley, Fall 2019, https://courses.d2l.ai/) using Pytorch. The textbook for this course is "Dive Into Deep Learning" (http://d2l.ai/).

Notebooks in this repo are more or less a practice coding session during the self-learning of the book and the course. So, compared to the original MxNet implementation, notebooks in this repo may contain more code comments, lecture notes, some personal options, and it is implemented entirely based on pytorch packages (does not need d2l package).

This repo also provides sample code to run some interesting algorithms mentioned in the book using Pytorch pretrained models:

• AlexNet
• VGG
• ResNet

This repo also compares some interesting differece between MxNet and PyTorch

• Tranfer between tensors/ndarray and numpy array
• Multi-GPU training
• Modify the number of class of output layer in pretrained model output

Syllabus

The structure of this repo follows STAT 157 syllabus.

• L01 Logistics, Software, Linear Algebra
• L02 Probability and Statistics (Bayes Rule, Sampling Naive Bayes, Sampling)
• L03 Gradients, Chain Rule, Automatic differentiation
• L04 Linear Regression, Basic Optimization
• L05 Likelihood, Loss Functions, Logisitic Regression, Information Theory
• L06 Multilayer Perceptron
• L07 Model Selection, Weight Decay, Dropout
• L08 Numerical Stability, Hardware
• L09 Environment
• L10 Layers, Parameters, GPUs
• L11 Convolutional Layers
• L12 LeNet, AlexNet, VGG, NiN
• L13 Inception, Residual Networks
• L14 Computation Performance, Multi-GPU and Multi-Machine Training
• L15 Image Augmentation, Fine Turning
• L16 Object Detection
• L18 Sequence models and Language
• L19 Recurrent neural networks
• L20 Bi-LSTM, Deep RNNs
• L22 Word2vec, FastText, GloVe, Sentiment Analysis
• L23 Encoder-Decoder, Seq2seq, Machine Translation
• L24 Attention, Transformer, BERT
• L25 Convex Optimization, Convergence Rate
• L26 Momentum, AdaGrad, RMSProp, Adam

Todo

• L15/5 Style Transfer
• L16/7 Single Shot Multibox Detection
• Faster RCNN
• Mask RCNN
• NLP models

Citation

If you find this work useful, please cite the original book:

@article{zhang2019dive,
  title={Dive into deep learning},
  author={Zhang, Aston and Lipton, Zachary C and Li, Mu and Smola, Alexander J},
  url={http://d2l.ai/},
  year={2019}
}