Giters

nlpxucan / WizardLM

LLMs build upon Evol Insturct: WizardLM, WizardCoder, WizardMath

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Number of vocab in WizardCoder-1B

hgKang02 opened this issue · comments

commented

Hi, I was wondering what the vocab size is for WizardCoder-1B.

I have tried to print the len(vocab) where vocab is from tokenizer.get_vocab() and it is printing 49153. Would this value be correct?

Would you be kind to tell me what the vocab size is? Thanks for your consideration.

Sincerely,
Hangoo Kang.