AMDonati / NLP-class-Emines-Jan22

Repository for a one-week NLP class given @ Emines, Mohammed VI Univesity, Ben Guerir

NLP-class-Emines-Jan22

Repository for a one-week NLP class given In January 2022 for the Data Science Specialty @ Emines, Mohammed VI University, Ben Guerir, Morocco.

Slides:

• Day 1: Introduction, Text Processing, word embeddings models
• Day 2: Recurrent Neural Networks, Application to Text Classification
• Day 3: Language Models, Natural Language Generation, Evaluation Metrics
• Day 4: Sequence to Sequence Models with Attention, Transformers - Application to Machine Translation
• Day 5: Pretrained Language Systems, Transfer Learning in NLP

Practical Exercises

• Day 1: Exploring Text preprocessing techniques and word embeddings models
• Day 2: Building a sentiment analysis classifier with a RNN on the Stanford Sentiment Treebank dataset
• Day 3: Training a RNN Language Model on the ROC Stories dataset
• Day 4: Build an end-to-end French-English Machine Translation System
• Day 5: Generating Text with pretrained Language Models
• Evaluated Exercises

Reading activities

• Day 1:

  1. GloVe embedding model
  2. Overview of word embeddings models: 1 + 2

• Day 2:

  1. Recap on RNNs
  2. Understand LSTMs
  3. Backpropagation through time for training LSTMs

• Day 3:

  1. N-gram Language Models
  2. The curious case of text degeneration
  3. Evaluation Metrics for Language Models

• Day 4:

  1. The Illustrated Transformer
  2. Visual Attention - Show, Attend and Tell

• Day 5:

  1. The Transformer XL - Learning Cross-Lingual embeddings
  2. Using BERT for topic modelling: 1 + 2

Additional ressources:

• Stanford NLP class
• Textbook - A primer on Neural Networks for Natural Language Processing (2015)
• Textbook - Speech and Language Processing (2021)
• Hugging face course on pretrained Language Models