GitHub

GitHub

BERT, Tokenizer, Python, WordPiece, pybind11,C++,Flash,Trie

4 followers

Visit website
🚀 **FlashTokenizer: World's Fastest CPU Tokenizer!** ⚡ 8~15x faster than `BertTokenizerFast` 🛠️ High-performance C++ 🔄 Parallel with OpenMP 📦 Easy pip install 💻 Cross-platform (Win/Mac/Linux) ▶️ Demo: https://youtu.be/a_sTiAXeSE0

GitHub makers

Here are the founders, developers, designers and product people who worked on GitHub

rowen
rowenA developer specializing in optimization
GitHubGitHub