jamesdellinger / fastai_deep_learning_course_part2_v3

My projects for fast.ai's Deep Learning from the Foundations course (fast.ai part2v3)

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Fast.ai Deep Learning from the Foundations (Spring 2019)

Part II of Fast.ai's two-part deep learning course, offered through The Data Institute at USF. From March through the end of April in 2019. My Part I coursework is here.

This course offered a bottom-up approach (through code, not math equations) to becoming an expert deep learning practitioner and experimenter.

We implemented core fastai and PyTorch classes and modules from scratch, achieving similar or better performance. We also practiced coding up techniques introduced in various papers, and then spent significant time on strategies useful in decreasing model training time (parallelization, JIT). The final two weeks were spent diving deep into Swift for TensorFlow with Chris Lattner, where we saw first-hand how differentiable programming could work.

I came away with both the know-how to engineer cutting-edge deep learning ideas from scratch with optimized code, as well as the expertise necessary to research and explore new ideas of my own.

My Explanations of Lesson Notebooks

I implemented the code created by Jeremy Howard and Sylvain Gugger for the course's weekly lectures and reproduced their results. However, the bulk of my time (about 5 months, full-time study) was spent crafting my own plain-English explanations of the techniques and concepts covered by the class. I'm proudest of my writing on language model pre-training and the Swift for TensorFlow framework.

Week 8: Building Optimized Matmul, Forward and Backpropagation from Scratch

Week 9: How to Train Your Model

Week 10: Wrapping up CNNs

Week 11: Data Loading, Optimizers, and Augmentations

Week 12: MixUp, XResNets, AWD-LSTM and ULMFiT

Weeks 13 and 14: Implementing fastai/PyTorch classes from scratch in Swift!

Papers We Studied

Week 8

Week 9

Week 10

Week 11

Week 12

Dependencies for PyTorch Notebooks

USF Completion Certificate

About

My projects for fast.ai's Deep Learning from the Foundations course (fast.ai part2v3)


Languages

Language:Jupyter Notebook 99.4%Language:Python 0.6%