makcedward / nlp

:memo: This repository recorded my NLP journey.

NLP - Tutorial

Repository to show how NLP can tacke real problem. Including the source code, dataset, state-of-the art in NLP

Data Augmentation

• Data Augmentation in NLP
• Data Augmentation library for Text
• Does your NLP model able to prevent adversarial attack?
• How does Data Noising Help to Improve your NLP Model?
• Data Augmentation library for Speech Recognition
• Data Augmentation library for Audio
• Unsupervied Data Augmentation
• Adversarial Attacks in Textual Deep Neural Networks
• Back Translation in Text Augmentation by nlpaug

General

• Tricks of Building an ML or DNN Model

Text Preprocessing

Section	Sub-Section	Description	Story
Tokenization	Subword Tokenization		Medium
Tokenization	Word Tokenization		Medium Github
Tokenization	Sentence Tokenization		Medium Github
Part of Speech			Medium Github
Lemmatization			Medium Github
Stemming			Medium Github
Stop Words			Medium Github
Phrase Word Recognition
Spell Checking	Lexicon-based	Peter Norvig algorithm	Medium Github
	Lexicon-based	Symspell	Medium Github
	Machine Translation	Statistical Machine Translation	Medium
	Machine Translation	Attention	Medium
String Matching	Fuzzywuzzy		Medium Github

Text Representation

Section	Sub-Section	Research Lab	Story	Source
Traditional Method	Bag-of-words (BoW)		Medium Github
	Latent Semantic Analysis (LSA) and Latent Dirichlet Allocation (LDA)		Medium Github
Character Level	Character Embedding	NYU	Medium Github	Paper
Word Level	Negative Sampling and Hierarchical Softmax		Medium
	Word2Vec, GloVe, fastText		Medium Github
	Contextualized Word Vectors (CoVe)	Salesforce	Medium Github	Paper Code
	Misspelling Oblivious (word) Embeddings	Facebook	Medium	Paper
	Embeddings from Language Models (ELMo)	AI2	Medium Github	Paper Code
	Contextual String Embeddings	Zalando Research	Medium	Paper Code
Sentence Level	Skip-thoughts		Medium Github	Paper Code
	InferSent		Medium Github	Paper Code
	Quick-Thoughts	Google	Medium	Paper Code
	General Purpose Sentence (GenSen)		Medium	Paper Code
	Bidirectional Encoder Representations from Transformers (BERT)	Google	Medium	Paper(2019) Code
	Generative Pre-Training (GPT)	OpenAI	Medium	Paper(2019) Code
	Self-Governing Neural Networks (SGNN)	Google	Medium	Paper
	Multi-Task Deep Neural Networks (MT-DNN)	Microsoft	Medium	Paper(2019)
	Generative Pre-Training-2 (GPT-2)	OpenAI	Medium	Paper(2019) Code
	Universal Language Model Fine-tuning (ULMFiT)	OpenAI	Medium	Paper Code
	BERT in Science Domain		Medium	Paper(2019) Paper(2019)
	BERT in Clinical Domain	NYU/PU	Medium	Paper(2019) Paper(2019)
	RoBERTa	UW/Facebook	Medium	Paper(2019) Paper
	Unified Language Model for NLP and NLU (UNILM)	Microsoft	Medium	Paper(2019)
	Cross-lingual Language Model (XLMs)	Facebook	Medium	Paper(2019)
	Transformer-XL	CMU/Google	Medium	Paper(2019)
	XLNet	CMU/Google	Medium	Paper(2019)
	CTRL	Salesforce	Medium	Paper(2019)
	ALBERT	Google/Toyota	Medium	Paper(2019)
	T5	Googles	Medium	Paper(2019)
	MultiFiT		Medium	Paper(2019)
	XTREME		Medium	Paper(2020)
	REALM		Medium	Paper(2020)

| Document Level | lda2vec | | Medium | Paper | | | doc2vec | Google | Medium Github | Paper |

NLP Problem

Section	Sub-Section	Description	Research Lab	Story	Paper & Code
Named Entity Recognition (NER)	Pattern-based Recognition			Medium
	Lexicon-based Recognition			Medium
	spaCy Pre-trained NER			Medium Github
Optical Character Recognition (OCR)	Printed Text	Google Cloud Vision API	Google	Medium	Paper
	Handwriting	LSTM	Google	Medium	Paper
Text Summarization	Extractive Approach			Medium Github
	Abstractive Approach			Medium
Emotion Recognition	Audio, Text, Visual	3 Multimodals for Emotion Recognition		Medium

Acoustic Problem

Section	Sub-Section	Description	Research Lab	Story	Paper & Code
Feature Representation	Unsupervised Learning	Introduction to Audio Feature Learning		Medium	Paper 1 Paper 2 Paper 3
Feature Representation	Unsupervised Learning	Speech2Vec and Sentence Level Embeddings		Medium	Paper 1 Paper 2
Feature Representation	Unsupervised Learning	Wav2vec		Medium	Paper
Speech-to-text		Introduction to Speeh-to-text		Medium

Text Distance Measurement

Section	Sub-Section	Description	Research Lab	Story	Paper & Code
Euclidean Distance, Cosine Similarity and Jaccard Similarity				Medium Github
Edit Distance	Levenshtein Distance			Medium Github
Word Moving Distance (WMD)				Medium Github
Supervised Word Moving Distance (S-WMD)				Medium
Manhattan LSTM				Medium	Paper

Model Interpretation

Section	Sub-Section	Description	Research Lab	Story	Paper & Code
ELI5, LIME and Skater				Medium Github
SHapley Additive exPlanations (SHAP)				Medium Github
Anchors				Medium Github

Graph

Section	Sub-Section	Description	Research Lab	Story	Paper & Code
Embeddings		TransE, RESCAL, DistMult, ComplEx, PyTorch BigGraph		Medium	RESCAL(2011) TransE(2013) DistMult(2015) ComplEx(2016) PyTorch BigGraph(2019)
Embeddings		DeepWalk, node2vec, LINE, GraphSAGE		Medium	DeepWalk(2014) node2vec(2015) LINE(2015) GraphSAGE(2018)
Embeddings		WLG, GCN, GAT, GIN		Medium	WLG(2011) GCN2017) GAT(2017) GraphSAGE(2018)
Embeddings		PinSAGE(2018)	Pinterest	Medium
Embeddings		HoIE(2015), SimpIE(2018)		Medium
Embeddings		ContE(2017), ETE(2017)		Medium

Meta-Learning

Section	Sub-Section	Description	Story
Introduction		Matching Nets(2016) MANN(2016) LSTM-based meta-learner(2017) Prototypical Networks(2017) ARC(2017) MAML(2017) MetaNet(2017)	Medium
NLP	Dialog Generation	DAML(2019), PAML(2019), NTMS(2019)	Medium
	Classification	Intent Embeddings(2016) LEOPARD(2019)	Medium
CV	Unsupervised Learning	CACTUs(2018)	Medium
General		Siamese Network(1994), Triplet Network(2015)	Medium
	MAML+(2018)	Medium

Image

Section	Sub-Section	Description	Research Lab	Story	Paper & Code
Object Detection		R-CNN		Medium	Paper(2013)
Object Detection		Fast R-CNN		Medium	Paper(2015)
Object Detection		Faster R-CNN		Medium	Paper(2015)
Object Detection		VGGNet		Medium	Paper(2014)
Instance Segmentation		Mask R-CNN	FAIR	Medium	Paper(2017)
Image Classification		ResNet(2015)	Microsoft	Medium
Image Classification		ResNeXt(2016)		Medium

Evaluation

Section	Sub-Section	Description	Story
Introduction			Medium
Classification		Confusion Matrix, ROC, AUC	Medium
Regression		MAE, MSE, RMSE, MAPE, WMAPE	Medium
Textual		Perplexity, BLEU, GER, WER, GLUE	Medium

Source Code

Section	Sub-Section	Description	Link
Spellcheck			Github
InferSent			Github