Rare find

Llama Tokenizer. Count tokens for Llama AI models right in your browser.

belladoreai.github.io/llama-tokenizer-js/example-demo/build

Vaya's read on this project

Problem, audience, market, and the verdict — sign in to see it.

Updates

June 2024
  • Release —v1.2.2
  • bump example-demo to 1.2.2.
  • Change behavior when user calls decode with invalid parameter combina…
April 2024
  • Example-demo bump to 1.2.1.
  • README.
  • README.
March 2024
  • README.
  • Release —v1.2.1
  • README.
  • TypeScript fix.
  • README.
  • README.
  • README.
  • README.
  • README.
  • Release —v1.2.0
  • Update demosite.
  • Various changes from blaze2004's PR.
November 2023
  • README.
  • README.
August 2023
  • Simplify mergePrio.
  • Release —v1.1.3
  • Update example-demo.
  • Support very large inputs.
  • Release —v1.1.2
June 2023
  • Release —v1.1.1
  • Release —v1.1.0
  • Release —v1.0.1
  • Release —v1.0.0