bpe-tokenizer-ts

🧠 Build and explore a minimal Byte Pair Encoding tokenizer in TypeScript, training and encoding text using raw UTF-8 bytes without external libraries.

GitHub

Official source

Start locally with the CLI guide