daodavid / classic-ML

Implementation of classic machine learning concepts and algorithms from scratch and math behind their implementation.Written in Jupiter Notebook Python

classic Machine Learning Algorithms

---

---

Linear Regression

• Simple Linear Regression


• Gradient Descent over simple linear regression


• Effect of different values for learning rate


• Multiple Linear Regression


• Implementation of gradient descent for Multiple Linear regression using NUMPY


• Test of our implemntation in 'insurance.csv' dataset


• The probabilistic approach to linear regression.Maximum likelihood estimation

---

---

Regularization

• Polynomial Regression, Bias and Variance


• Lasso Regression (L1 Regularization)


• Lasso as feature selection


• Ridge regression (L2 regularization)


• K-fold cross validation


• References

---

---

Logistic Regression

• Log-odds or Loggit function


• The math origin of the Sigmoid function


• Properties and Identities Of Sigmoid Function


• Maximum Likelihood of Logistic regression, Cross-entropy Loss


• Mathematical derivation of cross-entopy loss.Gradient Descent


• Implementation of BinaryLogisticRegression using numpy


• Reguralization of Logistic Regression


• References

---

---

Soft-Max Regression

• Abstract


• Softmaxt definition and how it works?


• Optimizaton of Softmax Loss with Gradient Descent (Deep math calculation)


• Implementation of Softmax using numpy


• Regularization of softmax by learning rate and max iterations


• Conclusion

---

---

Naive Bayes Classifier

• Bayes Theorem


• How does Binomial Naive Bayes work?


• Investigation of likelihood and posterior probability throw features values of Titanic dataset


• Implementation of likelihood table for Gaussian Naive Bayes and testing on Titanic


• Bernoulli Naive Bayes


• References