sampathsl / Keep-Learning

Resources and blogs for learning data science and machine learning. Pull requests and feedbacks are heartly welcomed

Keep Learning

Table of Contents

Software Engineering - ML	MLOps Core
MLOps: Infrastructure	Blog Resources For Machine Learning
MLOps: Model Deployment and Serving	MLOps: Testing, Monitoring and Maintenance
Blogs — Be Better Everyday	MIT 6S191 Introduction to Deep Learning
Master The Computer Vision — List of blogs and tutorials for diving deep into CV	Software Engineering — CRUX
Introduction to Computer Science	Dive Into Deep Learning
Convolutional Neural Networks - CS231n - Stanford University

MLOps Fundamentals

Software Engineering

• The Twelve Factors
• Book "Accelerate: The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations", 2018 by Nicole Forsgren et.al
• Book "The DevOps Handbook" by Gene Kim, et al. 2016
• State of DevOps 2019
• Clean Code concepts adapted for machine learning and data science.
• School of SRE

MLOps Core

• Machine Learning Operations: You Design It, You Train It, You Run It!
• MLOps SIG Specification
• ML in Production
• Awesome production machine learning: State of MLOps Tools and Frameworks
• Udemy “Deployment of ML Models”
• Full Stack Deep Learning
• Engineering best practices for Machine Learning
• 🚀 Putting ML in Production
• Stanford MLSys Seminar Series
• IBM ML Operationalization Starter Kit
• Productize ML. A self-study guide for Developers and Product Managers building Machine Learning products.
• MLOps (Machine Learning Operations) Fundamentals on GCP
• ML full Stack preparation
• Machine Learing Engineering in Production | DeepLearning AI

MLOps: Model Deployment and Serving

• AI Infrastructure for Everyone: DeterminedAI
• Deploying R Models with MLflow and Docker
• What Does it Mean to Deploy a Machine Learning Model?
• Software Interfaces for Machine Learning Deployment
• Batch Inference for Machine Learning Deployment
• AWS Cost Optimization for ML Infrastructure - EC2 spend
• CI/CD for Machine Learning & AI
• Itaú Unibanco: How we built a CI/CD Pipeline for machine learning with online training in Kubeflow
• 101 For Serving ML Models
• Deploying Machine Learning models to production — Inference service architecture patterns
• Serverless ML: Deploying Lightweight Models at Scale
• ML Model Rollout To Production. Part 1 | Part 2
• Deploying Python ML Models with Flask, Docker and Kubernetes
• Deploying Python ML Models with Bodywork

MLOps: Testing, Monitoring and Maintenance

• Building dashboards for operational visibility (AWS)
• Monitoring Machine Learning Models in Production
• Effective testing for machine learning systems
• Unit Testing Data: What is it and how do you do it?
• How to Test Machine Learning Code and Systems (Accompanying code)
• Wu, T., Dong, Y., Dong, Z., Singa, A., Chen, X. and Zhang, Y., 2020. Testing Artificial Intelligence System Towards Safety and Robustness: State of the Art. IAENG International Journal of Computer Science, 47(3).
• Multi-Armed Bandits and the Stitch Fix Experimentation Platform
• A/B Testing Machine Learning Models
• Data validation for machine learning. Polyzotis, N., Zinkevich, M., Roy, S., Breck, E. and Whang, S., 2019. Proceedings of Machine Learning and Systems
• Testing machine learning based systems: a systematic mapping
• Explainable Monitoring: Stop flying blind and monitor your AI
• WhyLogs: Embrace Data Logging Across Your ML Systems
• Evidently AI. Insights on doing machine learning in production. (Vendor blog.)
• The definitive guide to comprehensively monitoring your AI
• Introduction to Unit Testing for Machine Learning
• Production Machine Learning Monitoring: Outliers, Drift, Explainers & Statistical Performance
• Test-Driven Development in MLOps Part 1

MLOps: Infrastructure

• MLOps Infrastructure Stack Canvas
• Rise of the Canonical Stack in Machine Learning. How a Dominant New Software Stack Will Unlock the Next Generation of Cutting Edge AI Apps
• AI Infrastructure Alliance. Building the canonical stack for AI/ML
• Linux Foundation AI Foundation
• ML Infrastructure Tools for Production | Part 1 — Production ML — The Final Stage of the Model Workflow | Part 2 — Model Deployment and Serving
• The MLOps Stack Template (by valohai)

Resources — Popular Modern & Traditional Machine Learning Algorithms — Theory — Math — Implementation

• Machine Learning Cheatsheet — be used to with ML terms
• Deep Learning Book — helloworld_for_deep_learning
• Basic Image Processing — learn baics of image processing for image-preprocessing.
• Xgboost with Different Categorical Encoding Methods
• Linear Regression | Lasso Regression | Ridge Regession — details of regression concepts with thoery and code.
• Magic Behind, Gaussian Naive Bias Classification Algorithm
• The Theory and Code Behind K-Nearest Neighbors
• Learn About Decision Trees — Working and Methods in Layman's Term With Code
• Get Used With Logistic Regression — With Code and Math Running Behind This Algorithm
• Various Kinds of Distances in Data Mining and Machine Learning

Blogs — Be Better Everyday

Explore With Papers

• Efficient Estimation of Word Representations in Vector Space — Word2Vec
• eXtreme Gradient Boosting — A Scalable Tree Boosting System

Deep Learning

• Creating Neural Network From Scratch — Step By Step With Pythonic Code

• Learn About Bayesian Deep Learning

• Learn Neural Networks and Deep Learning From Scratch — Theory

• Learn BERT — Bidirectional Encoder Representations from Transformers — state-of-art NLP model

• Generative Pre-trained Transformer 3 (GPT-3) — revolutionary NLP model — 515 times more powerful than BERT

• XGBoost Tutorials — Docs from the creater themselves

Machinne Learning In Production

• ML Ops: Machine Learning as an Engineering Discipline
• Rules of Machine Learning : Best Practices for ML Engineering

Regular Expressions — Irksome, Yet Useful

• Regular Expression — Official Python Regex Module
• Learn Regex
• Regex Made Easy With Real Python

Numpy — Numerical Python

• Numpy Basics
• Numpy Fundamentals

Core Python

• Dive Into Python
• Learn About Python's Pathlib — No Really, Python's Pathlib is Great
• Python 101
• Object Oriented With Python — Wholesome Blog For Learning OOP with Python 3
• Code Refactoring for Software Engineering
• Guide to Python Design Patterns
• Popular Python Design Patterns - Explicitely Python
• Learn Python By Doing Python

Be Pythonic

• Writing Pythonic Code — Transforming from messy code to beautiful pythonic code
• Write More Pythonic Code
• PEP 8 -- Style Guide for Python Code
• The Hitchhiker’s Guide to Python!

Modern Machine Learning With Scikit-Learn

• Article: Stacking made easy with Sklearn
• Article: Curve Fitting With Python
• Article: A Guide to Calibration Plots in Python
• Calmcode: human-learn
• Datacamp: Supervised Learning with scikit-learn
• Datacamp: Machine Learning with Tree-Based Models in Python
• Datacamp: Introduction to Linear Modeling in Python
• Datacamp: Linear Classifiers in Python
• Datacamp: Generalized Linear Models in Python
• Notebook: scikit-learn tips
• Pluralsight: Building Machine Learning Models in Python with scikit-learn
• Video: human learn
• Youtube: dabl: Automatic Machine Learning with a Human in the Loop 00:25:43
• Youtube: Multilabel and Multioutput Classification -Machine Learning with TensorFlow & scikit-learn on Python
• Youtube: DABL: Automatic machine learning with a human in the loop- AI Latim American SumMIT Day 1

Pandas — Be Able to Manipulate Data

• Learn Basic EDA with Pandas
• Article: Modern Pandas
  • Modern Pandas | PART 1 | Introduction
  • Modern Pandas | PART 2 | Method Chaining
  • Modern Pandas | PART 3 | Indexes
  • Modern Pandas | PART 4 | Performance
  • Modern Pandas | PART 5 | Tidy
  • Modern Pandas | PART 6 | Visualization
  • Modern Pandas | PART 7 | Time Series
  • Modern Pandas | PART 8 | Scaling

Be able to implement models in Tensorflow and Keras

• Coursera: Introduction to Tensorflow
• Coursera: Convolutional Neural Networks in TensorFlow
• Deeplizard: Keras - Python Deep Learning Neural Network API
• Book: Deep Learning with Python (Page: 276)
• Datacamp: Deep Learning in Python
• Datacamp: Convolutional Neural Networks for Image Processing
• Datacamp: Introduction to TensorFlow in Python
• Datacamp: Introduction to Deep Learning with Keras
• Datacamp: Advanced Deep Learning with Keras
• Google: Machine Learning Crash Course
• Pluralsight: Deep Learning with Keras
• Udacity: Intro to TensorFlow for Deep Learning

Control Your Code — Versioning

• Version Control Via Git
• A Sucessful Git Branching Model
• Git & Github Crash Course
• Everything About Git & Gitbash

Developer Tools For ML

• Weight And Biases

Dive Into Deep learning

• Introduction
• Preliminaries

Convolutional Neural Networks

• Lecture 1 | Introduction to Convolutional Neural Networks
• Lecture 2 | Image Classification
• Lecture 3 | Loss Functions and Optimizations
• Lecture 4 | Introduction to Neural Networks
• Lecture 5 | Convolutional Neural Networks
• Lecture 6 | Training Neural Networks I
• Lecture 7 | Training Neural Networks II
• Lecture 8 | Deep Learning Software
• Lecture 9 | CNN Architectures
• Lecture 10 | Recurrent Neural Networks
• Lecture 11 | Detection and Segmentation
• Lecture 12 | Visualizing and Understanding
• Lecture 13 | Generative Models
• Lecture 14 | Deep Reinforcement Learning
• Lecture 15 | Efficient Methods and Hardware for Deep Learning
• Lecture 16 | Adversarial Examples and Adversarial Training

Master The Computer Vision — List of blogs and tutorials for diving deep into world of intelligent vision

Beginner Level — Mathematics

• Linear Algebra
• Singular Value Decomposition
• Basic Pattern Recognition
• Reduce The Dimesnion — PCA
• Guide To Kalman Filtering
• Fourtier Transforms

Beginner Level — Image Procesing

• Online Course offered by Duke University on Coursera
• Image Processing With Scipy and Numpy

Advanced Level

• Linear Discriminant Analysis
• Probability, Bayes rule, Maximum Likelihood, MAP
• Mixtures and Expectation-Maximization Algorithm
• Introductory level Statistical Learning
• Hidden Markov Models
• Support Vector Machines
• Genetic Algorithms
• Bayesian Networks

Theory Of Classical Machine Learning

• StatQuest: Machine Learning
  • StatQuest: Fitting a line to data, aka least squares, aka linear regression. 0:09:21
  • StatQuest: Linear Models Pt.1 - Linear Regression 0:27:26
  • StatQuest: StatQuest: Linear Models Pt.2 - t-tests and ANOVA 0:11:37
  • StatQuest: Odds and Log(Odds), Clearly Explained!!! 0:11:30
  • StatQuest: Odds Ratios and Log(Odds Ratios), Clearly Explained!!! 0:16:20
  • StatQuest: Logistic Regression 0:08:47
  • Logistic Regression Details Pt1: Coefficients 0:19:02
  • Logistic Regression Details Pt 2: Maximum Likelihood 0:10:23
  • Logistic Regression Details Pt 3: R-squared and p-value 0:15:25
  • Saturated Models and Deviance 0:18:39
  • Deviance Residuals 0:06:18
  • Regularization Part 1: Ridge (L2) Regression 0:20:26
  • Regularization Part 2: Lasso (L1) Regression 0:08:19
  • Ridge vs Lasso Regression, Visualized!!! 0:09:05
  • Regularization Part 3: Elastic Net Regression 0:05:19
  • StatQuest: Principal Component Analysis (PCA), Step-by-Step 0:21:57
  • StatQuest: PCA main ideas in only 5 minutes!!! 0:06:04
  • StatQuest: PCA - Practical Tips 0:08:19
  • StatQuest: PCA in Python 0:11:37
  • StatQuest: Linear Discriminant Analysis (LDA) clearly explained. 0:15:12
  • StatQuest: MDS and PCoA 0:08:18
  • StatQuest: t-SNE, Clearly Explained 0:11:47
  • StatQuest: Hierarchical Clustering 0:11:19
  • StatQuest: K-means clustering 0:08:57
  • StatQuest: K-nearest neighbors, Clearly Explained 0:05:30
  • Naive Bayes, Clearly Explained!!! 0:15:12
  • Gaussian Naive Bayes, Clearly Explained!!! 0:09:41
  • StatQuest: Decision Trees 0:17:22
  • StatQuest: Decision Trees, Part 2 - Feature Selection and Missing Data 0:05:16
  • Regression Trees, Clearly Explained!!! 0:22:33
  • How to Prune Regression Trees, Clearly Explained!!! 0:16:15
  • StatQuest: Random Forests Part 1 - Building, Using and Evaluating 0:09:54
  • StatQuest: Random Forests Part 2: Missing data and clustering 0:11:53
  • The Chain Rule 0:18:23
  • Gradient Descent, Step-by-Step 0:23:54
  • Stochastic Gradient Descent, Clearly Explained!!! 0:10:53
  • AdaBoost, Clearly Explained 0:20:54
  • Gradient Boost Part 1: Regression Main Ideas 0:15:52
  • Gradient Boost Part 2: Regression Details 0:26:45
  • Gradient Boost Part 3: Classification 0:17:02
  • Gradient Boost Part 4: Classification Details 0:36:59
  • Support Vector Machines, Clearly Explained!!! 0:20:32
  • Support Vector Machines Part 2: The Polynomial Kernel 0:07:15
  • Support Vector Machines Part 3: The Radial (RBF) Kernel 0:15:52
  • XGBoost Part 1: Regression 0:25:46
  • XGBoost Part 2: Classification 0:25:17
  • XGBoost Part 3: Mathematical Details 0:27:24
  • XGBoost Part 4: Crazy Cool Optimizations 0:24:27
  • StatQuest: Fiitting a curve to data, aka lowess, aka loess 0:10:10
  • Statistics Fundamentals: Population Parameters 0:14:31
  • Principal Component Analysis (PCA) clearly explained (2015) 0:20:16
  • Decision Trees in Python from Start to Finish 1:06:23

Video Tutorials — Deep Learning

• MIT 6S191 | Deep Learning Lectures

  • Lecture 1 | Indroduction to Deep Leaning
  • Lecture 2 | Recurrent Neural Networks
  • lecture 3 | Convolutional Neural Network
  • Lexture 4 | Deep Generative Models
  • Lecture 5 | Reinforcement Learning

• Machine Learning Engineering for Production (MLOps) Specialization — COURSERA SPECIALIZATION

  • Course I | Introduction to Machine Learning in Production
  • Course II | Machine Learning Data Lifecycle in Production
  • Course III | Machine Learning Modeling Pipelines in Production
  • Course IV | Deploying Machine Learning Models in Production

• MIT: 18.06 Linear Algebra (Professor Strang)

  • 1. The Geometry of Linear Equations 0:39:49
  • 2. Elimination with Matrices. 0:47:41
  • 3. Multiplication and Inverse Matrices 0:46:48
  • 4. Factorization into A = LU 0:48:05
  • 5. Transposes, Permutations, Spaces R^n 0:47:41
  • 6. Column Space and Nullspace 0:46:01
  • 9. Independence, Basis, and Dimension 0:50:14
  • 10. The Four Fundamental Subspaces 0:49:20
  • 11. Matrix Spaces; Rank 1; Small World Graphs 0:45:55
  • 14. Orthogonal Vectors and Subspaces 0:49:47
  • 15. Projections onto Subspaces 0:48:51
  • 16. Projection Matrices and Least Squares 0:48:05
  • 17. Orthogonal Matrices and Gram-Schmidt 0:49:09
  • 21. Eigenvalues and Eigenvectors 0:51:22
  • 22. Diagonalization and Powers of A 0:51:50
  • 24. Markov Matrices; Fourier Series 0:51:11
  • 25. Symmetric Matrices and Positive Definiteness 0:43:52
  • 27. Positive Definite Matrices and Minima 0:50:40
  • 29. Singular Value Decomposition 0:40:28
  • 30. Linear Transformations and Their Matrices 0:49:27
  • 31. Change of Basis; Image Compression 0:50:13
  • 33. Left and Right Inverses; Pseudoinverse 0:41:52

• Fast.ai: Deep Learning for Coder (2020)

  • Lesson 1
  • Lesson 2
  • Lesson 3
  • Lesson 4
  • Lesson 5
  • Lesson 6
  • Lesson 7
  • Lesson 8

• Full Stack Deep Learning

  • Setting Up Machine learning Projects
  • Infrastructure And Tooling
  • Data Management
  • Machine Learning Teams
  • Training And Debugging
  • Testing And Deployment

• CNN For Visual Recognition — cs231n

  • Lecture 1 | Introduction to Convolutional Neural Networks
  • Lecture 2 | Image Classification
  • lecture 3 | Loss Function and Optimization
  • lecture 4 | Introduction to Neural Networks
  • Lecture 5 | Convulutional Neural Network
  • Lecture 6 | Training Neural Network I
  • Lecture 7 | Training Neural Network II
  • Lecture 8 | Deep learning Software
  • Lecture 9 | CNN Architectures
  • Lecture 10 | Recurrent Neural Networks
  • Lecture 11 | Detection and Segmentation
  • Lecture 12 | Visualizing and Understanding
  • Lecture 13 | Generative Models
  • Lecture 14 | Deep Reinforcement Learning
  • Lecture 15 | Efficient Methods and Hardware for Deep Learning
  • Lecture 16 | Adversarial Examples and Adversarial Training

• Learn eXtreme Gradient Boosting - State-of-art ML Algorithm for Kaggle Contest till date.

  • Part I | Introduction and Regression with XGBoost
  • Part II | Classification with XGboost
  • Part III | Mathematical Details of XGBoost
  • Part IV | Cool Optimization with XGBoost

Introduction to Computer Science

• CS 10 - The Beauty and Joy of Computing - Spring 2015 - Dan Garcia - UC Berkeley InfoCoBuild
• 6.0001 - Introduction to Computer Science and Programming in Python - MIT OCW
• 6.001 - Structure and Interpretation of Computer Programs, MIT
• CS 50 - Introduction to Computer Science, Harvard University (cs50.tv)
• CS 61A - Structure and Interpretation of Computer Programs [Python], UC Berkeley
• CPSC 110 - Systematic Program Design [Racket], University of British Columbia
• CS50's Understanding Technology
• CSE 142 Computer Programming I (Java Programming), Spring 2016 - University of Washington
• CS 1301 Intro to computing - Gatech
• CS 106A - Programming Methodology, Stanford University (Lecture Videos)
• CS 106B - Programming Abstractions, Stanford University (Lecture Videos)
• CS 106X - Programming Abstractions in C++ (Lecture Videos)
• CS 107 - Programming Paradigms, Stanford University
• CmSc 150 - Introduction to Programming with Arcade Games, Simpson College
• LINFO 1104 - Paradigms of computer programming, Peter Van Roy, Université catholique de Louvain, Belgium - EdX
• FP 101x - Introduction to Functional Programming, TU Delft
• Introduction to Problem Solving and Programming - IIT Kanpur
• Introduction to programming in C - IIT Kanpur
• Programming in C++ - IIT Kharagpur
• Python Boot Camp Fall 2016 - Berkeley Institute for Data Science (BIDS)
• CS 101 - Introduction to Computer Science - Udacity
• 6.00SC - Introduction to Computer Science and Programming (Spring 2011) - MIT OCW
• 6.00 - Introduction to Computer Science and Programming (Fall 2008) - MIT OCW
• 6.01SC - Introduction to Electrical Engineering and Computer Science I - MIT OCW
• Modern C++ Course (2018) - Bonn University
• Modern C++ (Lecture & Tutorials, 2020, Vizzo & Stachniss) - University of Bonn

Software Engineering — CRUX

• Object Oriented Design
  • Object-oriented Program Design and Software Engineering - Aduni
  • OOSE - Object-Oriented Software Engineering, Dr. Tim Lethbridge
  • Object Oriented Systems Analysis and Design (Systems Analysis and Design in a Changing World)
  • CS 251 - Intermediate Software Design (C++ version) - Vanderbilt University
  • OOSE - Software Dev Using UML and Java
  • Object-Oriented Analysis and Design - IIT Kharagpur
  • CS3 - Design in Computing - Richard Buckland UNSW
  • Informatics 1 - Object-Oriented Programming 2014/15- University of Edinburgh
  • Software Engineering with Objects and Components 2015/16- University of Edinburgh
• Software Engineering
  • Computer Science 169- Software Engineering - Spring 2015 - UCBerkeley
  • CS 5150 - Software Engineering, Fall 2014 - Cornell University
  • Introduction to Service Design and Engineering - University of Trento, Italy
  • CS 164 Software Engineering - Harvard
  • System Analysis and Design - IISC Bangalore
  • Software Engineering - IIT Bombay
  • Dependable Systems (SS 2014)- HPI University of Potsdam
  • Software Testing - IIT Kharagpur
  • Informatics 2C - Software Engineering 2014/15- University of Edinburgh
• Software Architecture
  • CS 411 - Software Architecture Design - Bilkent University
  • MOOC - Software Architecture & Design - Udacity