openai / tiktoken

tiktoken is a fast BPE tokeniser for use with OpenAI's models.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

openai/tiktoken Watchers