lemonviv / DeepLearning

CS583: Deep Learning

1. Machine learning basics. This part briefly introduces the fundamental ML problems-- regression, classification, dimensionality reduction, and clustering-- and the traditional ML models and numerical algorithms for solving the problems.

   • ML basics. [slides-1] [slides-2]

   • Regression. [slides-1] [slides-2]

   • Classification.

     • Logistic regression: [slides] [lecture note]

     • SVM: [slides]

     • Softmax classifier: [slides]

     • KNN classifier: [slides]

   • Regularizations. [slides-1] [slides-2]

   • Clustering. [slides]

   • Dimensionality reduction. [slides-1] [slides-2] [lecture note]

   • Scientific computing libraries. [slides]

   • Monte Carlo. [slides] [Video (in Chinese)]

2. Neural network basics. This part covers the multilayer perceptron, backpropagation, and deep learning libraries, with focus on Keras.

   • Multilayer perceptron and backpropagation. [slides] [lecture note]

   • Keras. [slides]

   • Further reading:

     • [activation functions]

     • [parameter initialization]

     • [optimization algorithms]

3. Convolutional neural networks (CNNs). This part is focused on CNNs and its application to computer vision problems.

   • CNN basics. [slides]

   • Tricks for improving test accuracy. [slides]

   • Feature scaling and batch normalization. [slides]

   • Advanced topics on CNNs. [slides]

   • Popular CNN architectures. [slides]

   • Face recognition. [slides]

   • Further reading:

     • [style transfer (Section 8.1, Chollet's book)]

     • [visualize CNN (Section 5.4, Chollet's book)]

4. Recurrent neural networks (RNNs). This part introduces RNNs and its applications in natural language processing (NLP).

   • Categorical feature processing. [slides] [Video (in Chinese)]

   • Text processing and word embedding. [slides] [Video (in Chinese)]

   • RNN basics. [slides] [Video (in Chinese)]

   • LSTM. [slides] [reference] [Video (in Chinese)]

   • Making RNNs more effective. [slides] [Video (in Chinese)]

   • Text generation. [slides] [Video (in Chinese)]

   • Machine translation. [slides] [Video (in Chinese)]

   • Attention. [slides] [Video (in Chinese)] [reference-1] [reference-2]

   • Self-Attention. [slides] [Video (in Chinese)]

   • Image caption generation. [slides] [reference]

5. Language Models beyond RNNs.

   • Transformer (1/2): attention without RNN. [slides] [Video (in Chinese)]

   • Transformer (2/2): from shallow to deep. [slides] [Video (in Chinese)] [reference]

   • BERT: pre-training Transformer. [slides] [Video (in Chinese)] [reference]

6. Autoencoders. This part introduces autoencoders for dimensionality reduction and image generation.

   • Autoencoder for dimensionality reduction. [slides]

   • Variational Autoencoders (VAEs) for image generation. [slides]

7. Generative Adversarial Networks (GANs).

   • DC-GAN [slides]

8. Deep Reinforcement Learning.

   • Reinforcement learning [slides] [lecture note] [Video (in Chinese)]

   • Value-based learning [slides] [Video (in Chinese)]

   • Policy-based learning [slides] [Video (in Chinese)]

   • Actor-critic methods [slides] [Video (in Chinese)]

   • AlphaGo [slides] [Video (in Chinese)]

9. Parallel Computing.

   • Basics and MapReduce. [slides] [lecture note] [Video (in Chinese)]

   • Parameter Server and Decentralized Network. [slides] [Video (in Chinese)]

   • TensorFlow's Mirrored Strategy and Ring All-Reduce. [slides] [Video (in Chinese)]

   • Federated Learning. [slides] [Video (in Chinese)]

10. Adversarial Robustness. This part introduces how to attack neural networks using adversarial examples and how to defend from the attack.

    • Data evasion attack and defense. [slides] [lecture note]

    • Data poisoning attack. [slides] [Video (in Chinese)]

    • Further reading: [Adversarial Robustness - Theory and Practice]

11. Meta Learning.

    • Few-shot learning: basic concepts. [slides] [Video (in Chinese)] [Video (in English)]

    • Siamese network. [slides] [Video (in Chinese)]

    • Pretraining + fine tuning. [slides] [Video (in Chinese)]

12. Recommender System. This part is focused on the collaborative filtering approach to recommendation based on the user-item rating data. This part covers matrix completion methods and neural network approaches.

    • Collaborative filtering. [slides]