gpt-tokenizer. The fastest JavaScript BPE Tokenizer Encoder Decoder for OpenAI's GPT models (o1, o3, o4, gpt-4o, gpt-4, etc.). Port of OpenAI's tiktoken with additional features.

github.com/leogopal/gpt-tokenizer

Vaya's read on this project

Problem, audience, market, and the verdict — sign in to see it.

Updates

No recent activity.