Revision History
Vending-Bench · 5 revisions
Sizes are character counts of the article source. The signed number is the change from the previous revision.
Recent edit summaries
Detailed summaries recorded by editors. Generic maintenance summaries are omitted here; every recorded revision remains below. Dates describe the edit, not necessarily the event it covers.
- Improve internal links (2026-10-13)
Version 5 · Oct 13, 2026, 10:08 AM
- Correction: the third-place error bar is wider than the first-to-third gap, not first-to-fourth; AISI is the UK AI Security Institute; Epoch AI sources the public leaderboard; the deprecated board is ordered by minimum net worth
Version 4 · Oct 1, 2026, 01:18 PM
- Correction: the Arena misbehavior findings were Anthropic's report of Andon Labs' testing of Claude Opus 4.6 specifically (not "the highest-performing model"), run length is 2,000 messages with o3-mini reaching 222 simulated days (not "dozens"), runs consume about 25M tokens over 5-10 real hours, tool steps are 5/25/75 minutes or 5 hours, search is Perplexity, and the Inc. reference did not support the meltdown passage (replaced with the Andon Labs eval page). Update: Vending-Bench was deprecated on 18 November 2025; adds Vending-Bench 2 (methodology, system prompt, 1 Oct 2026 leaderboard with standard errors and run counts, trend fits, $63k headroom estimate), Vending-Bench Arena rounds 1-13, the per-model behavior analyses, and the Gemini 4 Argon result.
Version 3 · Oct 1, 2026, 10:31 AM
- Added 6 contextual internal links
Version 2 · Jul 23, 2026, 03:54 PM