Chapter 1: Intoruduction.
Run different one-liner transformer.pipelines with text example.
Chapter 2: Text classifier.
Train DistilBERT for sentiment analysis.Use two approaches:
- Extract text embedings from the last hidden layer, then train sklearn linear model
- Fine-tune DisltilBERT with the Huggingface Trainer
Chapter 3: Transformers anatomy.
Implement from scratch transformer's decoder.