bens1320 / NLP-Tutorials

Simple implementations of NLP models. Tutorials are written in Chinese on my website https://mofanpy.com

Natural Language Processing Tutorial

Tutorial in Chinese can be found in mofanpy.com.

This repo includes many simple implementations of models in Neural Language Processing (NLP).

All code implementations in this tutorial are organized as following:

1. Search Engine

• TF-IDF numpy / TF-IDF skearn

2. Understand Word

• Continuous Bag of Words (CBOW)
• Skip-Gram

3. Understand Sentence

• seq2seq
• CNN language model

4. All about Attention

• seq2seq with attention
• Transformer

5. Pretrained Models

• ELMo
• GPT
• BERT or my new attempt Bert with next mask

some of the results

TF-IDF visualization

word2vec visualization

seq2seq attention visualization

transformer encoder-decoder attention visualization

bert next masked attention visualization