GPT2 Quickly
Build your own GPT2 quickly, without doing many useless work.
This project is base on
You can try this project in colab right now.
Main file
βββ configs
β βββ test.py
β βββ train.py
βββ build_tokenizer.py
βββ predata.py
βββ predict.py
βββ train.py
Preparation
git clone git@github.com:mymusise/gpt2-quickly.git
cd gpt2-quickly
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt
0x00. prepare your raw dataset
this is a example of raw dataset: raw.txt
0x01. Build vocab
python build_tokenizer.py
0x02. Tokenize
python predata.py --n_processes=2
0x03 Train
python train.py
0x04 Predict
python predict.py
0x05 Fine-Tune
ENV=FINETUNE python finetune.py