How to work with Chinese？
SmileSmith opened this issue · comments
When select Chinese content result is something like bytes:\x8a
Are using gpt2 model? It might be the tokenizer issue. Will fix it soon.
Now you can specify the language you're working on. Choose zh to switch the base model from gpt2 to uer/gpt2-chinese-cluecorpussmall. Bug fixed.