zkry / tiktoken.el

tiktoken.el is an Emacs Lisp port of OpenAI's tiktoken library for BPE tokenization

Geek Repo

Github PK Tool