hpenedones / breadth-first-cs

A (table of contents of a) book covering Computer Science topics in a breadth first way

breadth-first-cs

A Table of Contents of a book that I dream of writing, covering Computer Science topics in a breadth first way, with a bias towards Machine Learning.

Machine Learning

• Supervised Learning

  • Classification
    • Linear Classifier
    • Decision Trees
      • Information Gain
    • Logistic Regression
    • Neural Networks
    • Support Vector Machines
    • Random Forests
  • Regression
    • Linear Regression and Least Squares
  • Ranking

• Ensemble Methods

  • Boosting
  • Bagging

• Unsupervised Learning

  • Clustering
    • K-means
    • EM for Gaussian Mixture Models
  • Manifold Learning and Dimensionality Reduction
    • PCA, Kernel PCA, ICA
    • Isomap an Local-Linear Embedding
    • Auto-encoders

• Deep Learning

  • Convolutional Neural Networks
  • Boltzmann Machines
  • AutoEncoders

• Reinforcement Learning

  • Bellman Equation
  • Multi-arm bandits
  • Exploration vs Exploitation

• Loss functions

  • Hinge Loss
  • MSE
  • Exponential Loss

• Regularisation

  • Lasso
  • Priors
  • Sparsity

• Overfitting

• Cross validation

• Model capacity

  • VC dimensionality

• Other topics

  • Hidden Markov Models
  • Conditional Random Fields
  • Monte Carlo
  • Gaussian Processes

• Optimisation

  • Gradient descent
  • Newton method
  • Stochastic Gradient Descent
  • Conjugate Gradient Descent
  • RMSProp
  • Convex optimisation for SVMs

• Robotics

  • Inverse Kinematics

• Statistics

  • Random Variables
  • Gaussian and Gaussian Mixture Models
  • Statistical Significance Tests
  • Maximum Likelihood
  • Sampling algorithms
  • Bayes Theorem

• Calculus

  • Differentiation
    • Chain rule
    • Product rule
    • Inverse
    • Power Rule
    • Reciprocal Rule (derived from the chain rule and the power rule)
    • Quotient Rule (derived from reciprocal rule and the product rule)
    • Derivative of Logarithm
  • Integrals
    • Integration by Parts
    • Integration by Substitution
  • Gradient
  • Jacobian matrix
  • Hessian matrix

• Combinatorics and Probabilities

  • Combinations, Permutations
  • Binomial theorem

• Linear Algebra

  • Matrix Multiplication
  • Matrix Inversion
  • Determinant
  • Solving linear system of equations
  • Eigen vectors and Eigen Values
  • Singular Value Decomposition
  • Covariance matrix
  • Positive-definite matrix
  • Conjugate transpose matrix
  • Hermitian matrix

• Information Theory

  • Entropy
  • Mutual Information
  • Conditional Entropy
  • Kullback Leibler Divergence
  • Kolmogorov Complexity

• Algorithms

  • Sorting
  • Graph shortest paths
  • BFS, DFS, A* search
  • Dynamic Programming
  • Shuffling
  • Local Sensitive Hashing

• Data Structures

  • Hash tables
  • Binary search trees
  • Balanced trees
  • Tries
  • Graphs

• Programming

  • Polymorphism
  • Virtual method
  • Inheritance
  • Encapsulation
  • Functional vs Imperative
  • Compiled vs Interpreted
  • Strongly typed vs Weekly typed

• Architecture

  • Scalability
  • Fault Tolerance
  • Modularity
  • Availability

• Distributed Systems and Parallelism

  • Map Reduce
  • GPUs
  • Threads, semaphores, deadlocks