catalyst-team / dl-course

Deep Learning with Catalyst

Deep Learning with Catalyst

This is an open deep learning course made by Deep Learning School, Tinkoff, and Catalyst team. Lectures and practice notebooks located in ./week* folders. Homeworks are in ./homework* folders.

Note: the course is under update: weeks with colab barge are ready to go, weeks with [WIP] label are still in progress. You could use the v20.12 branch for the earlier version of the full course.

Syllabus

• week 1: Deep learning intro
  • Deep learning – introduction, backpropagation algorithm. Optimization methods.
  • Neural Network in numpy.
• week 2: Deep learning frameworks
  • Regularization methods and deep learning frameworks.
  • Pytorch basics & extras.
• week 3: Convolutional Neural Network
  • CNN. Model Zoo.
  • Convolutional kernels. ResNet. Simple Noise Attack.
• week 4: Object Detection, Image Segmentation
  • Object Detection. (One, Two)-Stage methods. Anchors.
  • Image Segmentation. Up-scaling. FCN, U-net, FPN. DeepMask.
• week 5: Metric Learning
  • Metric Learning. Contrastive and Triplet Loss. Samplers.
  • Cross Entropy Loss modifications. SphereFace, CosFace, ArcFace.
• week 6: Autoencoders
  • AutoEncoders. Denoise, Sparse, Variational.
  • Generative Models. Autoregressive models.
• week 7: Generative Adversarial Models
  • Generative Adversarial Networks. VAE-GAN. AAE.
  • Energy based model.
• week 8: Natural Language Processing
  • Embeddings.
  • RNN. LSTM, GRU.
• week 9: Attention and transformer model
  • Attention Mechanism.
  • Transformer Model.
• week 10: Transfer Learning in NLP
  • Pretrained Transformers. BERT. GPT.
  • Data Augmentation in Texts. Domain Adaptation.
• week 11: Recommender Systems
  • Collaborative Filtering. FunkSVD.
  • Neural Collaborative Filtering.
• week 12: Reinforcement Learning for RecSys
  • DQN Algorithm.
  • DDPG Algorithm.
  • RecSim with Wolpertinger.
• [WIP] week 13: Extras
  • Research & Deploy.
  • Config API. Reaction.

Environment

Anaconda setup

# setup - env
conda create -n catalyst-dl python=3.7 anaconda
source activate catalyst-dl
conda remove nb_conda_kernels -y
conda install -c conda-forge nb_conda_kernels -y
conda install notebook jupyter nb_conda -y
conda remove nbpresent -y

# setup - jupyter
jupyter notebook password

# jupyter run
jupyter notebook --no-browser --ip 0.0.0.0 --port 8888

Requirements

pip install -U catalyst==21.04.2 torch==1.8.0 albumentations==0.5.0

Course staff & contributors

• @AlexeySh
• @artek0chumak
• @elephantmipt
• @Inkln
• @Fulldis
• @lordofprograms
• @Scitator
• @zelcookie