Revision History
SentencePiece · 7 revisions
Sizes are character counts of the article source. The signed number is the change from the previous revision.
Recent edit summaries
Detailed summaries recorded by editors. Generic maintenance summaries are omitted here; every recorded revision remains below. Dates describe the edit, not necessarily the event it covers.
- Correction: Corrected claims that DeepSeek and Qwen use SentencePiece (both use byte-level BPE tokenizers), removed an unsupported PaLM 2 tokenizer description and an invented rationale for T5's tokenizer choice, fixed the description of how SentencePiece BPE handles word boundaries, and removed a claim that Hugging Face Tokenizers is several times faster than SentencePiece, which SentencePiece's own benchmark contradicts (plus 19 minor fixes)
Version 7 · Nov 2, 2026, 11:25 AM
- Improve internal links (2026-10-13)
Version 6 · Oct 13, 2026, 10:18 AM
- Added 6 contextual internal links
Version 4 · Jul 23, 2026, 03:28 PM
- Pre-update snapshot before enhancement to 4664 words.
Version 2 · May 18, 2026, 12:08 AM