numbbbbb / tiktokenizer

Online playground for OpenAPI tokenizers

---

Tiktokenizer

Online playground for openai/tiktoken, calculating the correct number of tokens for a given prompt.

Special thanks to Diagram for sponsorship and guidance.

CleanShot.2023-03-02.at.22.58.11.mp4

Acknowledgments

• T3 Stack
• shadcn/ui
• openai/tiktoken