waleedgeo / Intro_Statistical_Learning

Introduction to Statistical Learning

Introduction to Statistical Learning

Registration Form for cohort 1: https://forms.gle/gRqq7uCp9FVcqgej9

Notes about the course:

The course Text books:

1. An Introduction to Statistical Learning: https://www.amazon.com/Introduction-Statistical-Learning-Applications-Statistics/dp/1461471370
2. Machine Learning: A Probabilistic Perspective https://www.amazon.com/Machine-Learning-Probabilistic-Perspective-Computation/dp/0262018020

Instructor : Omid Safarzadeh:

LinkedIn: https://www.linkedin.com/in/omidsafarzadeh/

IG : @deepdatascientists

Chapter 1 : Learning Algorithms

Supervised learning
Unsupervised learning
Semi-Supervised learning
Online Learning
Reinforcement Learning
Graph Representation Learning

Chapter 2 : Regression, Cross-Validation

Regression
The least squares approach
Multiple linear regression
Bias-variance tradeoff
Validation
Leave-one-out cross-validation
k-fold cross validation

Chapter 3 : Logistic , Ridge, Lasso Regression

logistic regression
    MLE for simple logistic regression 
Regularization
Ridge regression
Bias-variance tradeoff
Pros and cons of ridge regression

Chapter 4 : Bayesian Estimation, MAP

    Likelihood and posterior distribution
       Computing the posterior
       Maximum likelihood estimation (MLE)
    Maximum a posteriori (MAP) estimation
       Posterior mean
       MAP properties
    Bayesian linear regression

Chapter 5 : Unsupervised Learning, PCA

 Unsupervised learning 
 Principal component analysis (PCA)

Chapter 6 : Recommendation Systems

   Collaborative Filtering
   Matrix Factorization
   Funk SVD
   Alternating Least Square

Chapter 7 : EM Algorithm

 Expectation Maximization Algorithm

Chapter 8 : Clustering

Clustering
K means
Gaussian Mixture Models

Chapter 9: Activation and Loss functions

Activation Functions
    Exponential Linear Unit (ELU)
    Exponential activation function
    Gaussian error linear unit (GELU)
    Hard sigmoid
    Rectified Linear Unit (ReLU)
    Scaled Exponential Linear Unit (SELU)
    Sigmoid
    Softplus
    Softsign
    Swish
    Hyperbolic Tangent
Loss Functions
   Mean Absolute Error (MAE)
   Mean Absolute Percentage Error (MAPE)
   Mean Squared Error (MSE)
   Mean Squared Error (MSE)
   Indicator function

Chapter 10: Neural Networks, RNN, LSTM, CNN

    Gradient Decent
    Back Propagation
    CNN
    RNN
    LSTM

Chapter 11: Transformers

   Sequence to sequence models
   Attention Mechanism
      Bottleneck Problem
      Attention Layer
      Categories of Attention Mechanism
   Transformers
       Self attention mechanism
       Multi-Head attention mechanism
       Encoder Architecture
       Decoder Architecture
       Full Architecture
   Positional Encoding
   BERT
       Language Masked Learning
       BERT Input
       BERT Output

Chapter 12: Automatic Feature Extraction

 Motivation
 Automatic FE with TensorFlow
 Deep & Cross Network Structure
 Pre Processing
 Cross Network
    Deep NN
    Deep & Cross Network V1
    Deep & Cross Network V2
    Model construction
 Model understanding for interpreting cross features
 Model Performance