An introduction to deep learning with Python and Pytorch. Covers optmization, neural network basics, convolutional neural networks, and advanced topics such as autoencoders and generative adversarial networks.
2021 Instructor: Tomas Beuzen
Find the lecture schedule below. I'm developing new material for this course, but I've included links to old lectures and other useful videos for those that are interested.
| # | Topic | Optional Watching/Reading |
|---|---|---|
| 1 | Floating Point Errors |
|
| 2 | Optimization and Gradient Descent | |
| 3 | Stochastic Gradient Descent | |
| 4 | Introduction to Neural Networks & PyTorch | |
| 5 | Training Neural Networks | |
| 6 | Convolutional Neural Networks Part 1 | |
| 7 | Convolutional Neural Networks Part 2 | TBD |
| 8 | Advanced Neural Networks | TBD |
You are responsible for the following deliverables, which will determine your course grade:
| Assessment | Weight |
|---|---|
| Lab Assignment 1 | 15% |
| Lab Assignment 2 | 15% |
| Quiz 1 | 20% |
| Lab Assignment 3 | 15% |
| Lab Assignment 4 | 15% |
| Quiz 2 | 20% |
Labs are Jupyter notebooks comprised of more comprehensive exercises aimed at demonstrating and reinforcing concepts learned during lectures. Quizzes will be conducted on Canvas in week 3 and week 5, are open book and are typically 40 mins long with a focus on short-answer questions. More information on quizzes will be provided closer to their dates.
- Dive into Deep Learning, a book based on STAT 157 at UC Berkeley.
- Deep learning YouTube series by 3Blue1Brown.
- Neural Networks and Deep Learning (free online book).
- Deep Learning. Ian Goodfellow, Yoshua Bengio and Aaron Courville.
- Deep Learning with Python. Jason Brownlee.
- Stanford UFLDL tutorial (or here)
- Geoff Hinton Coursera lectures
- CS231n: Convolutional Neural Networks for Visual Recognition (Stanford)
- Grokking Deep Learning
- Practical Deep Learning For Coders, Part 1 and some more resources on their blog here
- A Guide to Deep Learning
- Awesome Deep Learning, which is a list of other resources
- Full Stack Deep Learning
- Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow: Concepts, Tools, and Techniques to Build Intelligent Systems by Aurélien Géron. Code/notebooks available here. (Endorsed by an MDS student!)
- James, Gareth; Witten, Daniela; Hastie, Trevor; and Tibshirani, Robert. An Introduction to Statistical Learning: with Applications in R. 2014. Plus Python code and more Python code.
- Russell, Stuart, and Peter Norvig. Artificial intelligence: a modern approach. 1995.
- David Poole and Alan Mackwordth. Artificial Intelligence: foundations of computational agents. 2nd edition (2017). Free e-book.
- Kevin Murphy. Machine Learning: A Probabilistic Perspective. 2012.
- Christopher Bishop. Pattern Recognition and Machine Learning. 2007.
- Pang-Ning Tan, Michael Steinbach, Vipin Kumar. Introduction to Data Mining. 2005.
- Mining of Massive Datasets. Jure Leskovec, Anand Rajaraman, Jeffrey David Ullman. 2nd ed, 2014.
- Mathematics for Machine Learning
- The Matrix Calculus You Need For Deep Learning
- Introduction to Optimizers
- Diabetic retinopathy Kaggle competition write-up
- Galaxy Zoo Kaggle competition write-up
- National Data Science Bowl competition write-up
Please see the general MDS policies.