Revision History
Multi-Head Self-Attention · 11 revisions
Sizes are character counts of the article source. The signed number is the change from the previous revision.
Recent edit summaries
Detailed summaries recorded by editors. Generic maintenance summaries are omitted here; every recorded revision remains below. Dates describe the edit, not necessarily the event it covers.
- Correction: Corrected the attention-head table (Llama 3.1 405B has 8 KV heads, PaLM's head size is 256, Falcon-40B has 128 heads of dimension 64), fixed the DeepSeek-V2 93.3% KV-cache comparison (it is against DeepSeek 67B, not same-backbone MHA), and replaced a BERT layer-hierarchy claim the cited Clark et al. paper does not make (plus 17 minor fixes)
Version 11 · Nov 2, 2026, 05:35 PM
- Improve internal links (2026-10-13)
Version 9 · Oct 13, 2026, 10:04 AM
- Added 6 contextual internal links
Version 8 · Jul 23, 2026, 03:02 PM
- Render formulas with LaTeX math notation
Version 7 · Jul 11, 2026, 05:26 PM