NLP Notes
Table of Contents
- Word embeddings
- RNN
- LSTM
- CNN for Text
- Attention and memory
- Seq2Seq
- t-sNE
Word embeddings
Basic Idea : Similair words tend to appear in similair context.
Generating word embeddings with a very deep architecture is simply too computationally expensive for a large vocabulary. This is the main reason why it took until 2013 for word embeddings to explode onto the NLP stage; computational complexity is a key trade-off for word embedding models and will be a recurring theme in our review.