AI Wiki

Explore AI Wiki

Citation and evidence

BridgeBench

14 min full readUpdated 26 references

This article's verification

Report a problem with this article

More

Use this article

Raw MarkdownExplore connections

Improve this page

Suggest editRevision history

Topics

AI BenchmarksAI Code GenerationModel Evaluation

Cite this article

BridgeBench is a website that publishes ratings and tests of large language models used for software development, describing itself as "independent benchmarks for AI coding models."[1] It is built and operated by BridgeMind, a company that sells tools for vibe coding and calls itself "the home of the vibe coding movement"; the site's footer carries a 2026 copyright notice for BridgeMind LLC, and its own guides say each bench is "part of BridgeBench, built by BridgeMind."[1][4][13] As of October 2026 the site has four benchmark sections: a leaderboard that rates 30 models on nine axes, Design Bench (a gallery of model-generated interactive scenes), Nerf Bench (repeated testing of a model after launch to detect performance changes) and Usage Bench (tracking of subscription usage allowances).[1][5][6] BridgeBench states that its benchmarking methodology for the leaderboard and Design Bench is proprietary and not disclosed, while Nerf Bench has a published methodology with private tasks.[3][9]

BridgeBench drew wide attention in April 2026, when BridgeMind posted that a retest showed Claude Opus 4.6 had been "nerfed." Critics pointed out that the earlier score came from 6 tasks and the retest from 30, and BridgeBench's own archived per-task results show little change on the tasks the two runs shared.[16][17][18][20]

Ownership and history

BridgeMind's X profile describes it as "the home of the vibe coding movement" and says it was founded by Matthew Miller, whose own profile lists him as founder and CEO of BridgeMind.[14] BridgeMind's main product is a desktop "Agent Super App" for Mac, Windows and Linux that runs coding agents such as Claude Code and Codex side by side, with users bringing their own AI subscription or API key; its Pro plan is listed at $50 per month.[13] BridgeBench's X account, which describes it as "the official AI benchmark of the vibe coding movement," was created in March 2026.[15]

Every BridgeBench page carries a sponsor slot. As of October 17, 2026, the slot advertised BridgeMind itself and invited bids "for the Oct 28 term from $1,500/month."[1]

DateEvent
March 21, 2026@bridgebench account created on X[15]
March 26, 2026BridgeBench v2 announced (the earliest dated post on the site's blog)[11]
April 2026Site lists separate UI, debugging, security, refactoring, reasoning, hallucination, "BS" and speed benchmarks, a DGX Spark hardware bench and community voting[17][19]
April 12, 2026BridgeMind posts that Claude Opus 4.6 "is nerfed" based on a hallucination retest[16]
September 22, 2026BridgeBench V3 announced, combining the nine-axis leaderboard, vendor API prices and Design Bench[5]
September 27, 2026Nerf Bench explainer published[4]
September 29, 2026Usage Bench's first measurements[6][7]
October 4, 2026Full Nerf Bench methodology published[3]

Expanded article table

Results from the v2 era are kept as archived blog posts. One records v2 ratings for GPT-5.4 and GPT-5.5 in debugging, reasoning, UI, refactoring and security, with the note that "these historical values are not comparable with current ratings."[12]

Leaderboard

The V3 leaderboard rates models on nine axes: reasoning, front end, back end, one shot, security, trustworthiness, efficiency, speed and cost, plus an overall rating.[5][9] BridgeBench describes the numbers as "comparative assessments, not success percentages" and says they "do not represent a head-to-head match record."[1][9] Some per-axis descriptions on model pages are explicitly judgments: the front-end rating for Claude Opus 5.5 is described as "BridgeBench's top editorial design pick," and the one-shot rating as "our estimate of first-pass completeness," not "a measured one-prompt success percentage."[10] The third-party aggregator BenchLM, which mirrors the leaderboard for display only, characterizes the figures as "editorial assessments expressed as Elo-style points" and notes that BridgeBench keeps its prompts, task sets and rubrics private.[23] Higher ratings are better on every axis, including cost, where a higher score means lower task spending.[1][10]

Top ten overall ratings as shown on October 17, 2026 (vendor list prices checked October 14):[1]

RankModelDeveloperOverallAPI price per 1M tokens (in / out)
1Claude Opus 5.5Anthropic760$4 / $20
2GPT-6 AstraOpenAI728$10 / $50
3Claude Fable 5.1Anthropic689$10 / $50
4Claude Sonnet 5.5Anthropic647$2 / $10
5GPT-6.1 SolOpenAI646$2 / $10
6GPT-6 SolOpenAI643$2 / $10
7Claude Fable 5Anthropic628$10 / $50
8Gemini 4 ArgonGoogle623$2 / $10
9Muse WonderMeta611$4 / $20
10Muse Spark 1.3Meta606$1.25 / $4.25

Expanded article table

The lowest-rated of the 30 models was GLM 5.3 Flash at 450.[1] Claude Opus 5.5's per-axis ratings illustrate the spread: 950 for front end, 920 for reasoning, 900 each for back end and trustworthiness, 870 for efficiency, 830 for one shot, 650 for security, 440 for cost and 380 for speed.[10]

Design Bench

Design Bench gives every model the same design prompt and publishes the outputs side by side, with the recorded generation cost and time for each.[1][8] In October 2026 it had five scenes (Lava Lamp, Rocket Launch, Sunset Ocean, Turntable and Black Hole Merger) and listed 27 models, 125 results and $40.29 in total generation spending.[8] Each output is marked as qualified, did not qualify, or not attempted. GPT-6 Astra, for example, qualified on four of five scenes, with its Turntable output marked "did not qualify."[8] The site says the cost and time figures describe that output only and are not estimates for other projects, and that "benchmarking details remain proprietary."[8][26]

Nerf Bench

Nerf Bench asks whether a model still performs as well as it did when BridgeBench started tracking it. The board defines a model's first test as 100% "launch power" and treats 90% to 110% as normal variance; below that range a model "has lost power since launch," above it the model is "stronger than at launch."[2]

Method

BridgeBench's methodology post, published October 4, 2026, gives the following details:[3]

  • Tasks: 50 private tasks in eight categories: bug fixing (15), code reasoning (10), code writing (5), algorithms (5), strings (5), testing (5), security (3) and SQL (2). The 33 code tasks are graded by running JavaScript against tests in a sandboxed Node process; the 17 answer tasks are graded by exact match after normalizing formatting. "No model grades another model's work."
  • Sessions: each session runs every task three times, for 150 pass-or-fail attempts. The model answers in a single response with no tools and no repository. The first complete session is frozen as the reference.
  • Score: power = 100 x (0.6 x S + 0.2 x K + 0.2 x C), where S is pass rate relative to the reference, K is output tokens (reference divided by session, so fewer tokens scores higher) and C is cost on the same basis. Each ratio is capped between 0 and 2. When cost data is missing, as for every Codex session, correctness is weighted 75% and tokens 25%.
  • Confirmation: a result outside the 90% to 110% band is published only if the immediately preceding comparable session was also outside the band on the same side.
  • Access paths: models are run through OpenRouter (default routing, no pinned provider, no temperature set), Claude Code and Codex, and each path is tracked as a separate series with its provider, model and effort setting locked after the first session. Each task allows up to 65,536 output tokens.
  • Cadence: "at least three sessions a week" for every tracked model, with extra sessions when users report a nerf.

The post justifies the band statistically: for a model passing 70% of attempts, chance alone moves power by up to about 9 points either way 95% of the time.[3] Because tokens and cost carry 40% of the weight, a model can lose power by becoming more verbose or expensive even if its answers are unchanged, and a price cut can raise power while the pass rate falls; BridgeBench says sessions like that are flagged for review.[3][4] BridgeBench also announced a $10,000 pledge "toward benchmarking over the coming months" so it can keep testing models after launch.[4]

Results (snapshot)

Four models were tracked on October 17, 2026, and none was outside the normal-variance band.[2]

ModelFirst test (100%)Lowest publishedLatest (Oct 16, 2026)
GPT-6 AstraSep 397.1% (Oct 16)97.1%
GPT-6.1 SolOct 198.0% (Oct 16)98.0%
Claude Sonnet 5.5Sep 2898.9% (Oct 16)98.9%
Claude Opus 5.5Sep 2294.2% (Oct 2)105.8%

Expanded article table

Claude Opus 5.5 had the widest range: it fell from 103.8% on October 1 to 94.2% on October 2, stayed between 95% and 97% in tests on October 4 and October 9, then rose to 105.8% on October 16, all inside the 90% to 110% band.[2][25] The board's published history shows fewer points than the stated cadence: GPT-6 Astra has a launch point on September 3 and its next on September 27, and every tracked model has points on October 9 and October 16 but none in between.[2] (The methodology says one complete session makes one chart point.)[3]

Usage Bench

Usage Bench tracks how much of a subscription plan's "day-one allowance" it still provides, separately for 5-hour and weekly usage limits. It is meant to answer whether "your plan got smaller," as distinct from Nerf Bench's question of whether a model got weaker.[6] Its first measurements, dated September 29, 2026, cover two plans: Claude Max 20x (5-hour and weekly limits, both 100%) and ChatGPT Pro (weekly limit 100%; the 5-hour limit is "not reported" because the plan shows none).[6][7] As of October 17, no retest had been published.[6] BridgeBench says the percentages compare each plan only with itself and do not rank plans against each other, and that the measurement methodology is proprietary.[7]

The April 2026 Opus 4.6 claim

On April 12, 2026, BridgeMind posted on X: "CLAUDE OPUS 4.6 IS NERFED. BridgeBench just proved it." The post said the model had ranked second on BridgeBench's hallucination benchmark the previous week with 83.3% accuracy, and that a retest that day placed it tenth with 68.3%, "a 98% increase in hallucination," adding that BridgeBench had "confirmed that Claude Opus 4.6 has reduced reasoning levels."[16] X showed the post at about one million views by October 2026, and it was quoted by, among others, Patrick Moorhead, who wrote that he had "experienced it."[16][24]

BridgeBench's own model pages, preserved by the Internet Archive, show what the two scores measured:

RunTasksAccuracyFabricationOverall score
Earlier run (captured April 12)[17]683.3%16.7%87.6
"April 12" retest[18]3068.3%33.0%73.3
"April 14" retest[19]3072.2%29.1%76.9

Expanded article table

Paul Calcraft, described by BeInCrypto as a computer scientist, called the claim "incredibly bad science," writing that the previous score was "on just 6 tasks," that the six tasks in common scored 85.4% in the retest against 87.6% before, and that the swing was "mostly from a single fabrication without repeats - easily statistical noise." BeInCrypto, reporting the exchange on April 13, concluded that "the BridgeBench data does not prove a deliberate downgrade."[20] The archived per-task tables bear out Calcraft's figures: on the six shared tasks the mean score was 87.6 in the earlier run and 85.4 in the April 12 retest, and five of the six tasks scored within about one point of their earlier result, with the difference coming mainly from a documentation task that dropped from 100 to 86.6 after one fabricated claim.[17][18]

On April 23, 2026, Anthropic published a postmortem tracing user reports that Claude's responses had worsened to three changes affecting Claude Code, the Claude Agent SDK and Claude Cowork: a change of Claude Code's default reasoning effort from high to medium (March 4 to April 7), a bug that repeatedly cleared Claude's earlier reasoning from sessions (March 26 to April 10), and a verbosity-reducing system prompt instruction (April 16 to April 20). All three affected Opus 4.6. Anthropic wrote that "the API was not impacted," that "we never intentionally degrade our models," and reset usage limits for all subscribers.[21] The postmortem does not mention BridgeBench. When Nerf Bench came up in a September 29, 2026 Hacker News discussion of the LiveNerf tracker, one commenter wrote that it "famously detected a degradation of Opus 4.6 which Anthropic later blogged about."[22] The April claim was based on BridgeBench's hallucination benchmark, not Nerf Bench, whose earliest published data point is GPT-6 Astra's launch test on September 3, 2026.[2][16]

Reception and limitations

  • Private methodology. BridgeBench withholds the methodology for its leaderboard and Design Bench "to protect the integrity of the results and reduce benchmark gaming," so its ratings cannot be reproduced independently.[9][23] Nerf Bench publishes its scoring formula and task categories but keeps prompts, tests and answers private "so they can't be trained on," a guard against data contamination.[3]
  • Small samples. The April 2026 dispute turned on comparing a 6-task run with a 30-task run, a weakness critics highlighted.[20] Nerf Bench's later design (150 attempts per session, a fixed reference and two-session confirmation) addresses single-run noise directly, though its own analysis shows that ordinary sampling can still move scores by close to 10 points.[3]
  • Ratings versus measurements. Leaderboard ratings combine axes such as speed and cost with capability axes, and some axes are described by the site as estimates or editorial picks rather than measured pass rates.[10][23]
  • Affiliation. BridgeBench describes itself as independent, but it is built by BridgeMind, which sells an agent workspace that runs Anthropic's and OpenAI's coding tools, and BridgeMind occupies the site's paid sponsor slot.[1][13]

See also

References

  1. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10BridgeBench. "Leaderboard". bridgebench.ai, accessed October 17, 2026. bridgebench.ai/leaderboard
  2. ^1 ^2 ^3 ^4 ^5BridgeBench. "Nerf Bench". bridgebench.ai, accessed October 17, 2026. bridgebench.ai/nerf-bench
  3. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8BridgeBench. "The Methodology of NerfBench". October 4, 2026. bridgebench.ai/...the-methodology-of-nerfbench
  4. ^1 ^2 ^3 ^4BridgeBench. "How Nerf Bench Detects When AI Models Get Nerfed". September 27, 2026 (updated October 4, 2026). bridgebench.ai/...how-nerf-bench-works
  5. ^1 ^2 ^3BridgeBench. "BridgeBench V3: The Vibe Coding Benchmark for AI Models". September 22, 2026. bridgebench.ai/...gebench-v3-vibe-coding-benchmark
  6. ^1 ^2 ^3 ^4 ^5BridgeBench. "Usage Bench". bridgebench.ai, accessed October 17, 2026. bridgebench.ai/usage-bench
  7. ^1 ^2 ^3BridgeBench. "How Usage Bench Tracks AI Subscription Limits". September 29, 2026. bridgebench.ai/...how-usage-bench-works
  8. ^1 ^2 ^3 ^4BridgeBench. "Design Bench". bridgebench.ai, accessed October 17, 2026. bridgebench.ai/ui-bench
  9. ^1 ^2 ^3 ^4BridgeBench. "A Guide to the AI Coding Leaderboard". September 2, 2026 (updated September 27, 2026). bridgebench.ai/...how-the-leaderboard-works
  10. ^1 ^2 ^3 ^4BridgeBench. "Claude Opus 5.5". bridgebench.ai, accessed October 17, 2026. bridgebench.ai/...claude-opus-5-5
  11. ^BridgeBench. "BridgeBench v2: An Archived Release". March 26, 2026. bridgebench.ai/...bridgebench-v2-release
  12. ^BridgeBench. "GPT-5.5 vs GPT-5.4: Archived BridgeBench Results". April 27, 2026. bridgebench.ai/...gpt-5-5-vs-gpt-5-4-bridgebench
  13. ^1 ^2 ^3BridgeMind. "BridgeMind" (home page). Accessed October 17, 2026. bridgemind.ai
  14. ^BridgeMind (@bridgemindai) and Matthew Miller (@matthewmillerai). X profiles, accessed October 17, 2026. x.com/bridgemindai
  15. ^1 ^2Bridgebench (@bridgebench). X profile, accessed October 17, 2026. x.com/bridgebench
  16. ^1 ^2 ^3 ^4 ^5BridgeMind (@bridgemindai). "CLAUDE OPUS 4.6 IS NERFED." X, April 12, 2026. x.com/...2043321284113670594
  17. ^1 ^2 ^3 ^4BridgeBench. "Hallucination Model Analysis: Claude Opus 4.6". Internet Archive capture, April 12, 2026. web.archive.org/...claude-opus-4-6
  18. ^1 ^2 ^3BridgeBench. "Hallucination Model Analysis: Claude Opus 4.6 (April 12)". Internet Archive capture, April 13, 2026. web.archive.org/...claude-opus-4-6-april-12
  19. ^1 ^2BridgeBench. "Hallucination Model Analysis: Claude Opus 4.6 (April 14)". Internet Archive capture, April 23, 2026. web.archive.org/...claude-opus-4-6-apr14
  20. ^1 ^2 ^3Okoth, Lockridge. "Viral BridgeBench Post Claims Claude Opus 4.6 Was 'Nerfed,' Critics Call It Bad Science". BeInCrypto, via Yahoo Tech, April 13, 2026. tech.yahoo.com/...nch-post-claims-claude-131318087
  21. ^Anthropic. "An update on recent Claude Code quality reports". April 23, 2026. anthropic.com/...april-23-postmortem
  22. ^Hacker News. "Livenerf: Has Opus 5.5 been nerfed yet?" (discussion thread), September 29, 2026. news.ycombinator.com/item
  23. ^1 ^2 ^3BenchLM. "BridgeBench". Accessed October 17, 2026. benchlm.ai/...bridgebench
  24. ^Moorhead, Patrick (@PatrickMoorhead). Post quoting BridgeMind. X, April 12, 2026. x.com/...2043419056162623740
  25. ^BridgeBench. "Nerf Bench". Internet Archive capture, October 3, 2026. web.archive.org/...nerf-bench
  26. ^BridgeBench. "A Guide to Design Bench Results". September 2, 2026 (updated September 27, 2026). bridgebench.ai/...how-ui-bench-works

Improve this article

Add missing citations, update stale details, or suggest a clearer explanation. Every suggestion is reviewed for sourcing before it goes live.

1 revision · v1 · 2,735 words · full history

Fact-checks are independent of edits: a reviewer re-verifies the article against its sources and stamps the date. How we verify

Research and drafting on this wiki are AI-assisted, under named human editorial standards. How AI is used here

Reviewer note: Independent verification 2026-10-17 (xg19 V6): all leaderboard ratings, Nerf Bench points and the April 2026 Opus 4.6 claim checked against live and archived pages; 0 defects. Snapshot data dated.

Cite this page: AI Wiki. "BridgeBench." aiwiki.ai, updated 16 Oct 2026, fact-checked 16 Oct 2026. CC BY 4.0. https://aiwiki.ai/wiki/bridgebench

Suggest edit

Where to go next

Topics:AI BenchmarksAI Code GenerationModel Evaluation

What links here

LiveNerf