Revision History
tiktoken · 7 revisions
Sizes are character counts of the article source. The signed number is the change from the previous revision.
Recent edit summaries
Detailed summaries recorded by editors. Generic maintenance summaries are omitted here; every recorded revision remains below. Dates describe the edit, not necessarily the event it covers.
- Correction: Corrected the README benchmark description (GPT-2 tokeniser, not cl100k_base), the cl100k_base vocabulary count and introduction date, the edit endpoint date, the default cache location, the cl100k_base regex, misattributed multilingual token figures and Modal/GPT-4o claims, the bpe crate benchmark, and README claims about training companions and offline use; added o200k_harmony and tiktoken 0.14.0 (plus 34 minor fixes)
Version 7 · Nov 5, 2026, 09:37 AM
- Improve internal links (2026-10-13)
Version 5 · Oct 13, 2026, 10:06 AM
- Added 6 contextual internal links
Version 4 · Jul 23, 2026, 03:46 PM