Revision History
ALiBi (Attention with Linear Biases) · 7 revisions
Sizes are character counts of the article source. The signed number is the change from the previous revision.
Recent edit summaries
Detailed summaries recorded by editors. Generic maintenance summaries are omitted here; every recorded revision remains below. Dates describe the edit, not necessarily the event it covers.
- Correction: Corrected the models and benchmarks attributed to the Cerebras ALiBi interpolation paper (BTLM-3B and MPT-7B on QMSum, GovReport and LongEval, not BLOOM-7B/Cerebras-GPT on SCROLLS) and an altered quote from it, replaced an unsupported claim that Google's Gemini Flash-Lite uses ALiBi with what the underlying study actually hypothesized, and fixed the RoPE speed and memory figures in the comparison table to match the ALiBi paper (plus 19 minor fixes)
Version 7 · Nov 5, 2026, 04:38 AM
- Update: added a video.
Version 6 · Nov 1, 2026, 08:26 AM
- Improve internal links (2026-10-13)
Version 5 · Oct 13, 2026, 10:04 AM
- Added 6 contextual internal links
Version 4 · Jul 23, 2026, 03:31 PM
- Initial AI-authored creation (3495 words).
Version 2 · May 19, 2026, 04:34 PM