AI Benchmarks

Explore AI Benchmarks through related topics and the articles other pages reference most.

Explore articles

Reset filters
Browse subtopics: Model Evaluation

Articles that also belong to these categories. Counts cover all of AI Benchmarks.

Showing 1-60 of 79 articles

AGIEval

AGIEval is an AI benchmark for evaluating foundation models on tasks that were originally designed for, and taken by, humans.

Model Evaluation

Agent evaluation

Agent evaluation is the systematic measurement of how well AI agents (LLM-based systems that plan and act over multiple steps using tools) perform on real-world tasks, using benchmarks, metrics, and testing…

AI AgentsModel Evaluation

Arena-Hard

Arena-Hard (and its evaluation tool Arena-Hard-Auto) is an automatic large language model (LLM) benchmark developed by the team behind Chatbot Arena that scores instruction-tuned models on 500 challenging

Model Evaluation

AutomationBench

AutomationBench is a benchmark for AI agents, built by the automation company Zapier, that tests whether a model can finish a multi-step business workflow across several simulated software applications.

AI AgentsModel Evaluation

Benchmark (AI)

In artificial intelligence and machine learning, a benchmark is a specified evaluation used to compare systems under common conditions.

Model Evaluation

BigFinanceBench

BigFinanceBench (BFB) is a benchmark for evaluating how well AI agents perform the research work of professional financial analysts, developed by the finance-AI company Rogo and announced on May 27, 2026.

Finance AIModel Evaluation

BlenderWorld

BlenderWorld is a benchmark for visual AI agents that build editable 3D scenes in Blender, the open-source 3D creation suite, from text briefs, reference images, reference videos and structured specification…

AI AgentsChinese AI

BridgeBench

BridgeBench is a website that publishes ratings and tests of large language models used for software development, describing itself as "independent benchmarks for AI coding models." It is built and operated by…

AI Code GenerationModel Evaluation

Build-A-Chip-Bench

Build-A-Chip-Bench is a benchmark for AI agents that tests whether an agent can carry a digital chip design from a workload specification all the way to a routed, verified GDS layout

AI AgentsAI Hardware

CaliBench

CaliBench is a benchmark for image-to-video generative models that asks whether a model reproduces the correct distribution of physical outcomes across many generations from the same starting frame

Model EvaluationVideo Generation

ChemBench

ChemBench is an automated AI benchmark that measures the chemical knowledge, reasoning, and safety judgment of large language models and compares their performance against expert human chemists.

Model Evaluation

CritPt

CritPt (Complex Research using Integrated Thinking - Physics Test, pronounced "critical point") is a benchmark that tests large language models on unpublished, research-level physics problems.

AI for ScienceLarge Language Models

Crusader Bench

Crusader Bench is an AI benchmark that tests AI agents by having them develop a settlement's economy in Stronghold Crusader: Definitive Edition.

AI AgentsAI in Gaming

Cybench

Cybench (short for Cybersecurity benchmark) is an open-source evaluation framework for measuring the cybersecurity capabilities and risks of large language model agents.

AI SafetyModel Evaluation

DesignArena

DesignArena (also written Design Arena) is a crowdsourced benchmark and consumer creation platform for AI-generated design, built by the San Francisco startup Intelligence.

AI CompaniesModel Evaluation

EnigmaEval

EnigmaEval is an AI benchmark of long, complex multimodal puzzles drawn from real-world puzzle hunts, designed to measure the unstructured, creative, multi-step reasoning abilities of frontier AI models.

Model Evaluation

FormulaOne

FormulaOne is an AI benchmark for evaluating whether a model can design and implement dynamic programming algorithms for graph problems.

Model Evaluation

GDP.pdf

GDP.pdf is a benchmark from Surge AI that tests whether multimodal language models can answer realistic professional questions about real PDF documents, such as benefits packets, leases, datasheets, clinical…

Enterprise AIModel Evaluation

GDPx

GDPx is an AI benchmark published by SemiAnalysis on November 2, 2026 to evaluate agents working in simulated professional environments.

AI AgentsModel Evaluation

HELMET

HELMET (How to Evaluate Long-context Models Effectively and Thoroughly) is a benchmark for evaluating long-context language models introduced by researchers at Princeton University and Intel Labs in 2024.

Model Evaluation

HalluLens

HalluLens is a large language model hallucination benchmark introduced by researchers at Meta AI's Fundamental AI Research (FAIR) lab, together with collaborators at the Hong Kong University of Science and…

Meta AIModel Evaluation

KernelBench

KernelBench is an AI benchmark and open-source evaluation environment that measures how well large language models can write fast and correct GPU kernels.

Model Evaluation

LAB-Bench

LAB-Bench (the Language Agent Biology Benchmark) is an AI benchmark of more than 2,400 multiple-choice questions built to measure how well language models and AI agents can perform practical biology research…

Model Evaluation

LM Evaluation Harness

LM Evaluation Harness (the Language Model Evaluation Harness, often abbreviated lm-eval or written lm-evaluation-harness) is an open-source software framework for measuring the performance of large language…

Model EvaluationOpen Source AI

LMArena

LMArena is a crowdsourced artificial intelligence evaluation platform and company that ranks large language models by having anonymous users vote on which of two blind, side-by-side model responses they prefer

AI CompaniesModel Evaluation

LiveNerf

LiveNerf (styled livenerf in its repository) is an open-source, pre-registered benchmark that tracks whether a frontier large language model gets worse after it is released.

AnthropicModel Evaluation

MATH

MATH is a benchmark of 12,500 competition mathematics problems used to evaluate the mathematical problem-solving ability of machine learning systems, particularly large language models.

Model EvaluationReasoning Models

MMLU-Redux

MMLU-Redux is a manually re-annotated, error-corrected subset of the Massive Multitask Language Understanding (MMLU) benchmark

Model Evaluation

MRCR

MRCR (Multi-Round Co-reference Resolution) is a synthetic long-context evaluation that tests whether a large language model can locate and disambiguate among several near-identical "needles" buried inside a…

Large Language ModelsModel Evaluation

MuSR

MuSR (Multistep Soft Reasoning) is a benchmark for evaluating multistep reasoning in large language models, built around long free-text narratives such as murder mysteries, object-placement scenarios, and…

Model EvaluationReasoning Models

MultiLangBench

MultiLangBench is an AI coding benchmark introduced by OpenAI on October 19, 2026. It evaluates implementations of a SQLite-compatible database engine across 20 programming languages.

Model Evaluation

NoLiMa

NoLiMa, short for "No Literal Matching," is a long-context benchmark for large language models that measures how well a model can find and use a single relevant fact buried in a long document when that fact…

Large Language ModelsModel Evaluation

OlympiadBench

OlympiadBench is an AI benchmark of 8,476 Olympiad-level mathematics and physics problems, designed to test the advanced scientific reasoning of large language models and large multimodal models.

Model Evaluation

Omni-MATH

Omni-MATH is an AI benchmark of Olympiad-level competition mathematics, introduced in October 2024 to measure the mathematical reasoning ability of large language models on problems far harder than those in…

Model Evaluation

Pass@k

Pass@k is the standard metric for evaluating code generation models: it measures the probability that at least one of k generated candidate solutions passes all of a problem's unit tests.

AI Code GenerationMachine Learning

PutnamBench

PutnamBench is a benchmark for evaluating automated theorem proving systems, especially neural and large-language-model-based provers, on competition mathematics.

Model Evaluation