midinas / ml-readings

List of machine learning resources I found most valuable

ml-readings

List of machine learning resources I found most valuable

Graphical Models

• Graphical Models, Exponential Families, and Variational Inference, Jordan, Wainwright
• Classic HMM Intro Paper Rabiner
• Introduction to Markov Random Fields

Probabilistic Modeling

• Introduction to Kalman- and Bayesian-Filters (Book as a jupyter notebook)
• Bayesian Modeling in Python
• Monte Carlo method intro Video, Iain Murray NIPS2015
• MCMC Sampling intro/tutorial
• Online Book on Probabilistic Generative Models, Goodman, Tenenbaum
• Intro/Intuition on Variational Inference (e.g. KLqp vs. KLpq dist.div.) D.MacKay Lecture
• Advanced Inference in Graphical Models - Youtube Lectures J.Bilmes
• Edward python lib / Paper
• pyMC3 for Bayesian Modeling
  • Python/PyMC3 versions of the programs described in Doing bayesian data analysis by John K. Kruschke

Gaussian Processes

• Neil Lawrence Lectures/Notebooks
  • http://nbviewer.jupyter.org/github/gpschool/gprs15a/blob/master/gaussian%20process%20introduction.ipynb

NN

• Harnessing Deep Neural Networks with Logic Rules
• Zero-Shot Learning for Semantic Utterance Classification - 2015 Dauphin et al.

Introductions to Convolutional neural networks

• CNN Stanford introduction
• colah's blog on convolutions
• WILDML blog on CNN for NLP problems

RNN / Attention

• Neural Machine Translation - seq2seq, ICLR 2015 Bahdanau, Cho, Bengio
• Grammar as a foreign language, 2014 Vinyals, Kaiser, Koo, Petrov, Sutskever, Hinton
• Pointer Networks, 2015 Vinyals, Fortunato, Jaitly
• Neural GPUS Learn Algortihms - 2016 Kaiser, Sutskever (Multilayer CGRU - generalizing to 2000bit length ops w/o error)

Generative / GAN

• Video Frame Generation via Cross Convolution
• OpenAI: Generative Adversarial Imitation Learning
• Unsup. feature disentangling - InfoGAN
• Wasserstein GAN
• The numerics of GANs / Consensus Optimization

Reinforcement Learning

• Evolution Strategies as a Scalable Alternative to Reinforcement Learning
• DeepMind paper on deep RL (Deep Q-Learning, experience replay)
• Deep Reinforcement Learning with Double Q-learning

NLP / Q&A / Memory Networks

• Predicting distributions with Linearizing Belief Networks
• Weakly supervised memory networks, 2015 Sukhbaatar, Szlam, Weston, Fergus

MISC

• Arxiv Sanity Preserver
• Stanford Unsupervised Feature Learning and Deep Learning
  • Backpropagation and its vectorization
  • Gradient checking
  • Network initialization/activations explained, Stanford CS231nWinter, Karpathy
• ML @ Stanford cheatsheet
• NIPS2014 Workshop list/videos
• Online Book D.McKay: Information Theory,Inference,and Learning Algorithms
• What is the intuition behind a beta distribution
• sugartensor lib on top of tensorflow (e.g. Quasi-RNN reference impl)
• Python

Datasets

• Stanford Qestion&Answering Dataset SQuAD

Other Books / Tutorials

• Probability Theory - The Logic of Science, E.T. Jaynes

Blogs

• colah's blog
  • intuition on information/entropy/KL divergence
• InFERENCe

Computer Science

• Power sets

Statistics

• Intro to confidence intervals and hypothesis testing