CS583: Deep Learning
-
Machine learning basics. This part briefly introduces the fundamental ML problems-- regression, classification, dimensionality reduction, and clustering-- and the traditional ML models and numerical algorithms for solving the problems.
-
Classification.
-
Logistic regression: [slides] [lecture note]
-
SVM: [slides]
-
Softmax classifier: [slides]
-
KNN classifier: [slides]
-
-
Clustering. [slides]
-
Dimensionality reduction. [slides-1] [slides-2] [lecture note]
-
Scientific computing libraries. [slides]
-
Monte Carlo. [slides] [Video (in Chinese)]
-
Neural network basics. This part covers the multilayer perceptron, backpropagation, and deep learning libraries, with focus on Keras.
-
Multilayer perceptron and backpropagation. [slides] [lecture note]
-
Keras. [slides]
-
Further reading:
-
-
Convolutional neural networks (CNNs). This part is focused on CNNs and its application to computer vision problems.
-
CNN basics. [slides]
-
Tricks for improving test accuracy. [slides]
-
Feature scaling and batch normalization. [slides]
-
Advanced topics on CNNs. [slides]
-
Popular CNN architectures. [slides]
-
Face recognition. [slides]
-
Further reading:
-
[style transfer (Section 8.1, Chollet's book)]
-
[visualize CNN (Section 5.4, Chollet's book)]
-
-
-
Recurrent neural networks (RNNs). This part introduces RNNs and its applications in natural language processing (NLP).
-
Categorical feature processing. [slides] [Video (in Chinese)]
-
Text processing and word embedding. [slides] [Video (in Chinese)]
-
RNN basics. [slides] [Video (in Chinese)]
-
LSTM. [slides] [reference] [Video (in Chinese)]
-
Making RNNs more effective. [slides] [Video (in Chinese)]
-
Text generation. [slides] [Video (in Chinese)]
-
Machine translation. [slides] [Video (in Chinese)]
-
Attention. [slides] [Video (in Chinese)] [reference-1] [reference-2]
-
Self-Attention. [slides] [Video (in Chinese)]
-
-
Language Models beyond RNNs.
-
Transformer (1/2): attention without RNN. [slides] [Video (in Chinese)]
-
Transformer (2/2): from shallow to deep. [slides] [Video (in Chinese)] [reference]
-
BERT: pre-training Transformer. [slides] [Video (in Chinese)] [reference]
-
-
Autoencoders. This part introduces autoencoders for dimensionality reduction and image generation.
-
Generative Adversarial Networks (GANs).
- DC-GAN [slides]
-
Deep Reinforcement Learning.
-
Reinforcement learning [slides] [lecture note] [Video (in Chinese)]
-
Value-based learning [slides] [Video (in Chinese)]
-
Policy-based learning [slides] [Video (in Chinese)]
-
Actor-critic methods [slides] [Video (in Chinese)]
-
AlphaGo [slides] [Video (in Chinese)]
-
-
Parallel Computing.
-
Basics and MapReduce. [slides] [lecture note] [Video (in Chinese)]
-
Parameter Server and Decentralized Network. [slides] [Video (in Chinese)]
-
TensorFlow's Mirrored Strategy and Ring All-Reduce. [slides] [Video (in Chinese)]
-
Federated Learning. [slides] [Video (in Chinese)]
-
-
Adversarial Robustness. This part introduces how to attack neural networks using adversarial examples and how to defend from the attack.
-
Data evasion attack and defense. [slides] [lecture note]
-
Data poisoning attack. [slides] [Video (in Chinese)]
-
Further reading: [Adversarial Robustness - Theory and Practice]
-
-
Meta Learning.
-
Few-shot learning: basic concepts. [slides] [Video (in Chinese)]
-
Siamese network. [slides] [Video (in Chinese)]
-
Pretraining + fine tuning. [slides] [Video (in Chinese)]
-
-
Recommender System. This part is focused on the collaborative filtering approach to recommendation based on the user-item rating data. This part covers matrix completion methods and neural network approaches.
- Collaborative filtering. [slides]