Deep learning course
This repo supplements Deep Learning course by Victor Lempitsky, taught at YSDA and Skoltech @spring'18. For previous iteration visit the fall17 branch.
Lecture and seminar materials for each week are in ./week* folders. Homeworks are in ./homework* folders.
General info
- Create cloud jupyter session from this repo -
- Telegram chat room (russian).
- YSDA deadlines & admin stuff can be found at the YSDA course wiki (ysda students only).
- Any technical issues, ideas, bugs in course materials, contribution ideas - add an issue
Syllabus
-
week01 Intro to deep learning
- Lecture: Deep learning -- introduction, backpropagation algorithm
- Seminar: Neural networks in numpy
- Homework 1 is out!
-
week02 Adaptive optimization methods
- Lecture: Empirical risk minimization, standard loss functions, linear classification, stochastic optimizers, adaptive SGD
- Seminar: Adaptive optimizers in numpy
- Please begin worrying about installing pytorch. You will need it next week!
-
week03 Convolutional networks I
- Lecture: Convolutional networks (ConvNets), computer vision
- Seminar: Symbolic graphs (pytorch),
- Homework 2 is out!
-
week04 Convolutional networks II
- Lecture: ConvNet architectures, representations inside CNNs; visualizing networks/inceptionism, transfer learning
- Seminar: Fine-tuning a pre-trained network
-
week05 Advanced Computer vision
- Lecture: "Deep" computer vision beyond classification; Verification tasks, object detection architectures, semantic segmentation
- Seminar: Semantic segmentation
- Homework 3 is out!
-
week06 Deep generative models I
- Lecture: Deep image generation; generative ConvNets, perceptual loss functions.
- Seminar: Art Style Transfer by Dmitry Ulyanov
-
week07 Deep generative models II
- Lecture: Generative Adversarial Networks
- Seminar: Generative Adversarial Networks
-
week08 Unsupervised deep learning
- Lecture: Autoencoders, variational autoencoders, image analogies
- Seminar: Variational autoencoders
-
week09 Deep learning for natural language processing
- Lecture: Word embeddings, word2vec and other variants, convolutional networks for natural language
- Seminar: Word embeddings. Text convolutions for salary prediction.
- Homework 4 is out!
-
week10 Recurrent neural networks
- Lecture: Modelling sequences. Simple RNN. Why BPTT isn't worth 4 letters. GRU/LSTM.
- Seminar: Generating human names and deep learning papers with RNNs
-
week11 Recurrent neural networks II
- Lecture: Sequence2sequence, architectures with attention and long-term memory.
- Seminar: Image Captioning
-
week12: Deep Reinforcement Learning
- Lecture: Reinforcement Learning, MDPs, policy gradient methods
- Seminar: REINFORCE on simple robot control, optional: advantage actor-critic on atari
Contributors & course staff
Course materials and teaching performed by
- Victor Lempitsky - all lectures
- Victor Yurchenko - intro notebooks, admin stuff
- Vadim Lebedev - notebooks, admin stuff
- Dmitry Ulyanov - notebooks on generative models & autoencoders
- Fedor Ratnikov - pytorch & nlp notebooks
- Oleg Vasilev - notebooks, technical issue resolution
- Arseniy Ashukha - image captioning materials
- Mikhail Khalman - variational autoencoder materials