kyrcha / ml-directory

Link collection for various resources on machine learning

ml-directory

Machine learning directory

Papers

Reference papers

• Batch normalization: Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Dropout regularization: Dropout: A Simple Way to Prevent Neural Networks from Overfitting
• Elastic net regularization: Regularization and variable selection via the elastic net

Books

Statistics

• Think Stats: Probability and Statistics for Programmers by Allen Downey - (with free PDF download link)

Reinforcement Learning:

• Reinforcement Learning: An Introduction by Sutton & Barto

Courses

Machine Learning:

• Learning from Data

Deep stuff:

• CS231n: Convolutional Neural Networks for Visual Recognition
• CS224d: Deep Learning for Natural Language Processing
• CS 294: Deep Reinforcement Learning

ML Heuristics

Quotes

• Keep PCA components to maintain 97-98% of variance (Andrew Ng, First ML MOOC)
• k=10 in k-fold CV (Ambroise McLachlan)
• Use 10x samples the VC-dimension for training (Learning from Data course by Yaser S. Abu-Mostafa)
• Word2vec: Skip Gram works well with small amount of data and is found to represent rare words well. On the other hand, CBOW is faster and has better representations for more frequent words. Article

Papers

Papers that account heuristics:

• A Sensitivity Analysis of (and Practitioners' Guide to) Convolutional Neural Networks for Sentence Classification
• An Empirical Exploration of Recurrent Network Architectures
• LSTM: A Search Space Odyssey
• An overview of gradient descent optimization algorithms

Pages

• CNN Tricks
• How to Train a GAN? Tips and tricks to make GANs work
• Guideline to select the hyperparameters in Deep Learning
• ML Fundamentals Checklist by Microsoft

Topics

AutoML - Automated Machine Learning:

• AutoML.org
• AutoML Workshops:
  • 2014 @ICML
  • 2015 @ICML
  • 2016 @ICML
• MetaSel Workshop 2014 - Meta-Learning and Algorithm Selection @ECAI

Web sites

• http://deeplearning.net/
• Data Science Trello public board

Tutorials

• Deep Learning tutorials directory
• Kaggle tutorials directory
• Unsupervised Feature Learning and Deep Learning
• Geoffrey Hinton's Neural Networks for Machine Learning
• General Sequence Learning using Recurrent Neural Networks
• Hacker's guide to Neural Networks
• Deep Learning tutorial @ Kaggle
• Andrew W. Moore tutorials on several aspects of statistical data mining.

CNN

• CNN for NLP
• A Comprehensive Guide to Convolutional Neural Networks

RNN

• RNN tutorial in 4 parts
• The Unreasonable Effectiveness of Recurrent Neural Networks
• Understanding LSTMs

Software

• https://www.r-project.org/ [R]
• http://www.scipy.org/ [Python]
• TensorFlow
• pandas - Python Data Analysis Library
• xgboost

Recommender Systems:

• Surprise - A Python scikit for building and analyzing recommender systems

Deep Learning:

• Theano
• DL4J
• Caffe
• torch

Blog roll

• Andrej Karpathy