bpe-cpp. implementation of Byte-Pair Encoding (BPE) for subword tokenization, written entirely in C++ . The tokenizer learns merges from raw text and supports encoding/decoding with UTF-8

github.com/mrinalxdev/bpe-cpp

Vaya's read on this project

Problem, audience, market, and the verdict — sign in to see it.

Updates

No recent activity.