Revision History
Rectified Linear Unit (ReLU) · 14 revisions
Sizes are character counts of the article source. The signed number is the change from the previous revision.
Recent edit summaries
Detailed summaries recorded by editors. Generic maintenance summaries are omitted here; every recorded revision remains below. Dates describe the edit, not necessarily the event it covers.
- Correction: Corrected the AlexNet 'six times faster' attribution (it was measured on a four-layer CIFAR-10 network, not AlexNet) and the 10.8-point margin (10.9), the claim that OpenAI's GPT series uses GELU throughout (gpt-oss uses SwiGLU), and an overstated description of Arora et al.'s depth-efficiency theorem (plus 12 minor fixes)
Version 14 · Nov 4, 2026, 04:33 PM
- Update: added a video.
Version 13 · Nov 1, 2026, 08:15 AM
- Start the article with its lead paragraph: removed the opening "Rectified Linear Unit (ReLU)" heading. No text changed.
Version 12 · Oct 29, 2026, 09:24 AM
- Added an explanatory diagram with caption and image credits
Version 11 · Oct 2, 2026, 08:56 PM
- Added 6 contextual internal links
Version 10 · Jul 23, 2026, 03:02 PM