bobflagg / cs224d-deep-learning-for-nlp

Notes, code, and other resoruces for auditing the course CS224d: Deep Learning for Natural Language Processing

cs224d-deep-learning-for-nlp

Notes, code, and other resources for auditing the course CS224d: Deep Learning for Natural Language Processing.

Lectures

• Lecture 2: Simple Word Vector representations: word2vec, GloVe [slides] [video]

• word2vec

• GloVe: Global Vectors for Word Representation

• Lecture 3: Advanced word vector representations: language models, softmax, single layer networks [slides] [video]

• Efficient Estimation of Word Representations in Vector Space. Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. In Proceedings of Workshop at ICLR, 2013.

• Distributed Representations of Words and Phrases and their Compositionality. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado, and Jeffrey Dean. In Proceedings of NIPS, 2013.

• GloVe: Global Vectors for Word Representation. Jeffrey Pennington, Richard Socher, Christopher D. Manning. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing.

• Improving Word Representations via Global Context and Multiple Word Prototypes. Eric H. Huang, Richard Socher, Christopher D. Manning, Andrew Y. Ng. 2012.

• The WordSimilarity-353 Test Collection

• Improving Distributional Similarity with Lessons Learned from Word Embeddings, Omer Levy, Yoav Goldberg, and Ido Dagan. TACL 2015.

Other Resources

• Making sense of word2vec
• Improving Distributional Similarity with Lessons Learned from Word Embeddings, Omer Levy, Yoav Goldberg, and Ido Dagan. TACL 2015.
• Getting Started with Word2Vec and GloVe
• Word2vec Tutorial, 2014, Radim Řehůřek
• Optimizing word2vec in gensim
• A GloVe implementation in Python - foldl
• Modern Methods for Sentiment Analysis, Michael Czerny

Blue Sky

• Weakly Supervised Memory Networks. Sainbayar Sukhbaatar, Arthur Szlam Jason Weston. 2015.
• When Are Tree Structures Necessary for Deep Learning of Representations?, 2015, Jiwei Li, Dan Jurafsky, Eudard Hovy

Extensions

• Scaling Recurrent Neural Network Language Models, Will Williams, Niranjani Prasad, David Mrva, Tom Ash, Tony Robinson
• Embedding-Based Word Similarity
• Linguistic Regularities in Sparse and Explicit Word Representations, 2014, Omer Levy and Yoav Goldberg
• word2vec
• Embedding-Based Word Similarity
• Getting Started with Word2Vec and GloVe

LSTM

• LSTM Networks for Sentiment Analysis
• LSTM (Long Short Term Memory)

Recurrent Neural Networks

Theano

• Keras: Theano-based Deep Learning library

General

• Deep Learning University

• GloVe - Global Vectors for Word Representation - Pennington, Socher, Manning 2014

What's Wrong with Deep Learning?

• What's wrong with convolutional nets?, 2014, Geoffrey Hinton
• Optimizing Neural Networks that Generate Images, 2014, Tiijman Tieleman
• Transforming Auto-encoders, 2011, G. E. Hinton, A. Krizhevsky & S. D. Wang