dada325 / Machine-Learning-Principle

The revisit of the Machine learning and Deep learning fundamentals and implementation Code

Machine-Learning-Algorithm-overview

The revisit of the Machine learning and Deep learning fundamentals and implementation Code

Alright! Let's have some fun with the Machine Learning. This is the code accompany with the blog posts series.

1. Mathematical Foundations:

   • Gaussian Distribution
   • Exponential Family Distributions
   • Confronting the Partition Function

2. Supervised Learning Algorithms:

   • Linear Algorithms:
     • Linear Regression
     • Logistic Regression
     • Linear Classification
   • Tree-Based Algorithms:
     • Decision Trees
     • Random Forest
     • Gradient Boosted Trees (e.g., XGBoost, LightGBM, CatBoost)
   • Neural Networks:
     • Feedforward Neural Networks
     • Convolutional Neural Networks (CNNs)
     • Recurrent Neural Networks (RNNs)
     • Long Short Term Memory (LSTM)
     • Transformers (e.g., BERT, GPT)
   • Support Vector Machines (SVM) and Kernel Methods:
     • SVM
     • Kernel Methods
   • Bayesian Methods:
     • Bayesian Linear Regression
     • Gaussian Process Regression
   • K-Nearest Neighbors (KNN)
   • Naive Bayes
   • Discriminant Analysis:
     • Linear Discriminant Analysis (LDA)
     • Quadratic Discriminant Analysis (QDA)

3. Unsupervised Learning Algorithms:

   • Clustering:
     • K-Means
     • Hierarchical Clustering
     • DBSCAN
     • Gaussian Mixture Models (GMM)
     • Spectral Clustering
   • Dimensionality Reduction:
     • Principal Component Analysis (PCA)
     • t-Distributed Stochastic Neighbor Embedding (t-SNE)
     • Autoencoders
     • Dimensionality Reduction (generic)
   • Generative Models:
     • Generative Adversarial Network (GAN)
     • Variational Autoencoder (VAE)
     • Restricted Boltzmann Machine (RBM) & Deep Boltzmann Machine (DBM)
     • Sigmoid Belief Network & Deep Belief Network (DBN)
     • Generative Models (generic)
   • Probabilistic Graphical Models:
     • Probabilistic Graphical Models (generic)
     • Hidden Markov Model (HMM)
     • Conditional Random Field (CRF)
     • Linear Dynamical Systems
     • Gaussian Networks

4. Semi-Supervised and Unsupervised Learning:

   • Variational Inference
   • Markov Chain Monte Carlo (MCMC)
   • Approximate Inference
   • Particle Filtering
   • Normalizing Flow
   • Expectation-Maximization (EM) Algorithm
   • Self-training
   • Pseudo-labeling

5. Reinforcement Learning Algorithms:

   • Q-Learning
   • Deep Q Networks (DQN)
   • Actor-Critic
   • Proximal Policy Optimization (PPO)
   • Advantage Actor Critic (A2C)
   • Monte Carlo Tree Search (MCTS)

6. Ensemble Methods:

   • Bagging (Bootstrap Aggregating)
   • Boosting
   • Stacking

7. Anomaly Detection:

   • Isolation Forest
   • One-Class SVM

8. Recommendation Systems:

   • Collaborative Filtering
   • Content-Based Filtering
   • Matrix Factorization (e.g., Singular Value Decomposition, SVD)
   • Hybrid Methods

9. Time Series Analysis:

   • Autoregressive Integrated Moving Average (ARIMA)
   • Prophet
   • State Space Models

10. Optimization Algorithms:

    • Gradient Descent
    • Stochastic Gradient Descent (SGD)
    • Adam, RMSprop, etc.

11. Regularization Methods:

    • Ridge Regression (L2 regularization)
    • Lasso Regression (L1 regularization)
    • Elastic Net