Qwen2 is the large language model series developed by Qwen team, Alibaba Cloud.
Geek Repo:Geek Repo
Github PK Tool:Github PK Tool
yileld opened this issue a month ago · comments
If I use tokenizer.id_to_token(), the text is very strange, but if I use tokenizer.decode() to decode ids, the result is right. Are these two method different?