ThuraAung1601 / NLP-Course

Basic concepts which are used in NLP such as : Language model, Neural Language model, Word embedding, Text classification, Bert, RNN, LSTM, GRU, Attention, Transformers.

Natural Language Processing

Project

Implemented a chatbot , using intent detection model which undrestands user intention and slotfilling model to get information from user query and will recommend us a resturant based on our query.

Homeworks

week 1

• Unigram Language model, Bigram Language model, N-gram Language model, Zipfs law, Words probability using Ngram models

week 2

• Bayesian smoothing with dirichlet prior, Perplexity of sentence and corpus, trigram neural language model using feed forward network

week 3

• gensim word2vec and doc2vec, TSNE for visualizing high dimensional plots, TF-IDF implementation, find document similarity using word2vec weighted mean average by tf-idf and doc2vec model

week 4

• Hazm and parsivar Library, Levenshtein distance calculation

week 5

• Word2Vec model, RNN, LSTM, GRU, Attention, f1-score, Accuracy, Recall, Precision

week 6

• Seq2Seq models, Dialogue systems, Transformers, Encoder-decoders.

week 7

• Transformer, Bert Tokenizer, Retrival based chatbot

Course material

week 1

• Class: Lingusitics knowledge, NLP challenges, Probabilistic Language modeling, Word token vs. Word type

week 2

• Class: Smoothing, Laplace smoothing, backoff and interpolation, Bayesian smoothing with drichlet prior, Absolute discounting, Kneser-ney Smoothing, Bayesian smoothing based on pitman-yor process, Entropy, Perplexity

  Check this article:

  1. Neural language model

week 3

• Class: Spare word representation, Dense word representation, term-document matrix, cosine similarity, tf-idf, positive point wise mutual information, SVD, brow clustering, skip-gram and CBOW, GLOVE

  Check this articke:

  1. Word Embedding
  2. Introduction to Word Embedding and Word2Vec
  3. Difference between Word2vec and onehot representation

week 4

• Class: Tokenization, Normalization, Lemmatization, Stemming, Stopword removal, Minimum edit distance,

week 5

• Class: Recurrent neural networks, Vanishing graident descent, LSTM, GRU, Bidirectional RNN, Bidirectional LSTM, Bidirectional GRU, Transformers, Attention.

  Check these articles:

  1. RNN, LSTM, GRU

  2. Encoder Decoder model

  3. Attention, Self attention (transformers)

  4. Evaluation metrics

  5. Difference between word2vec and embedding layer

week 6

• Class: Dialogue systems, general chatbots (Conversational agent), tasked-based chatbots, rule base or corpus base conversational agents, Turning, Speech act, grounding, subdialogues, initative, inference, Eliza and Parry as Rule base conversational agent, information retrival in corpus base, Neural text matching, Representation base model, interaction base model, hybrid model, Generation methods.

  Check these articles:

  1. Transformers

  2. Transformer architecture

  3. Attention architecture

  4. Why attentions

week 7

• Class: Contextualized representation, Elmo, Bert, masked language model, next sentence prediction, Bert family, GPT.

  Check these articles:

  1. Statistic vs Contextualizer word representation

  2. GPT models

  3. BERT

  4. BERT family

Notes

Check my class notes here.