How to use GPT2 tokenizer
LeoXinhaoLee opened this issue · comments
Thank you so much for releasing this amazing repo! I'm wondering how I can use GPT-2 tokenizer in your data processing pipeline. Thank you very much for your time and help.
This is no longer urgent. Thank you.