behnazj / Stanford-icme-workshop-2020

Files & lecture slides from summer workshop covering topics from machine learning to HCP [note copyright]

Information

This Repository contains files and lecture slides from icme 2020 summer workshops and covered a wide range of topics from machine learning to HCP. Some documents are publicly available from instructors' websites (copyright notice).

• Workshop #0: Stat
• Workshop #1: Py
• Workshop #2: ML
• Workshop #3: DL
• Workshop #4: HCP

Additional Resources

Statistics

• Session 1
  • Descriptive statistics for exploring data, especially visualization
  • Sampling, studies and experiments
• Session 2
  • Probability: basic rules, conditional probability, Bayes' rule
  • Sampling distributions and the central limit theorem
• Session 3
  • Regression
  • Confidence intervals and the bootstrap principle
• Session 4
  • Tests of significance, multiple comparisons and reproducibility

Here is an additional resource:

• Statistics Visualizations - Seeing Theory

Machine Learning

• Session 1 Introduction
  • Overview of Machine Learning
• Session 2 Unsupervised Learning
  • Clustering (K-means, Hierarchical Clustering)
  • Dimensionality Reduction (PCA, ICA, MDS, SOM, tSNE)
  • Imputation
• Session 3
  • Supervised Learning: Principles
    • Cross-validation
    • Regularization and Sparsity (lasso, ridge regression, elastic net)
• Session 4
  • Supervised Learning: Methods
    • Classification and Regression Trees
    • Ensembles
    • Neural

Reference books (see reference folder):

1. An Introduction to Statistical Learning with Applications in R by G. James, D. Witten, T. Hastie, and R. Tibshirani - Free online book
2. The Elements of Statistical Learning by T. Hastie, R. Tibshirani, and J. Friedman - Free online book

Deep Learning

• Session 1
  • Introduction
  • Current state of the art in deep learning
  • Math review
  • Architecture of multi-layer neural networks
• Session 2
  • Loss functions
  • The backpropgation algorithm
  • The gradient descent algorithm
  • Over-fitting and Under-fitting
• Session 3
  • Convolutional Neural Networks
  • Recurrent Neural Networks
  • Other Architectures
  • Deep Learning Libraries
  • Hands-on coding Session - Tensorflow
• Session 4
  • Hands-on coding Session - Keras
  • Hands-on coding Session - Transfer Learning
  • Failures of deep learning

Here are some additional resources for various topics:

• Books
  • Neural Networks and Deep Learning by Michael Nielsen - Free online book
  • Deep Learning by Ian Goodfellow and Yoshua Bengio and Aaron Courville
• Visualizations
  • Neural Network Playground - A playground for dense neural networks
  • Gan Lab - A playground for GANs
  • Initializing neural networks - Visual tutorial on initialization in deep learning
  • Parameter optimization in neural networks - Visual tutorial on optimization in deep learning
• Stanford Courses
  • CS 230 Deep Learning
  • CS 231N Convolutional Neural Networks for Visual Recognition
  • CS 224N Natural Language Processing with Deep Learning
  • CS 236 Deep Generative Models
• Interesting talks on advanced topics
  • Ben Recht - Training on test set and other heresies
  • Aleksander Madry - A new perspective on Adversarial Perturbations

High Performance Computing

• Session 1: Algorithms
• Session 2: Shared Memory
• Session 3: Distributed Memory
• Session 4: Spark