RSI-Exam
RSI-Exam is a benchmark that tests whether an AI agent can take a working but weak executable method, improve it through hours of autonomous experimentation, and produce a version that still performs better on…
AI Benchmarks
Explore AI Benchmarks through related topics and the articles other pages reference most.
Explore articles
Browse subtopics (51)
Articles that also belong to these categories. Counts cover all of AI Benchmarks.
All articles (239)Model Evaluation (56)Natural Language Processing (42)Large Language Models (33)AI Agents (28)AI Code Generation (25)AI Safety (25)Machine Learning (22)Computer Vision (19)Multimodal AI (15)Reasoning Models (13)Data & Datasets (11)Artificial Intelligence (8)OpenAI (6)AI Research (5)Healthcare AI (4)Mathematics (4)Open Source AI (4)AI Companies (3)AI Ethics (3)AI Hardware (3)Developer Tools (3)Google DeepMind (3)Image Generation (3)Information Retrieval (3)NVIDIA (3)Robotics (3)Reinforcement Learning (2)Software Development (2)Speech & Audio AI (2)Statistics (2)World Models (2)2025 in artificial intelligence (1)AI Alignment (1)AI Inference (1)AI Models (1)AI in Cybersecurity (1)Autonomous Vehicles (1)Data Science (1)Deep Learning (1)Education AI (1)Embodied AI (1)Enterprise AI (1)Finance AI (1)Generative AI (1)Legal AI (1)Meta AI (1)Microsoft (1)People (1)Physical AI (1)Research Organizations (1)Video Generation (1)
Showing 181-239 of 239 articles
RULER (benchmark)
RULER is a synthetic benchmark from NVIDIA that measures the real, usable context window of large language models (LLMs) by testing them on 13 tasks across four categories (retrieval, multi-hop tracing…
Real-SWE
Real-SWE is a coding-agent benchmark published in September 2026 by Specific Labs, a San Francisco company that licenses operational data and source code from businesses and packages them as training and…
RepoBench
RepoBench is an AI benchmark for repository-level code auto-completion, introduced in the 2023 paper "RepoBench: Benchmarking Repository-Level Code Auto-Completion Systems" by Tianyang Liu, Canwen Xu, and…
RewardBench
RewardBench is a benchmark and public leaderboard for evaluating reward models, the scoring functions that sit at the center of reinforcement learning from human feedback (RLHF).
SIMPLER
SIMPLER (Simulated Manipulation Policy Evaluation for Real Robot Setups) is a collection of simulated robot manipulation environments, released in 2024
SQuAD
SQuAD (the Stanford Question Answering Dataset) is a large-scale reading comprehension benchmark from Stanford University in which a model must answer a question by extracting the exact span of text that…
SUPERB
SUPERB, which stands for Speech processing Universal PERformance Benchmark, is a comprehensive evaluation framework designed to measure how well self-supervised learning (SSL) models generalize across a…
SWE-Atlas
SWE-Atlas is a benchmark for evaluating AI coding agents on professional software-engineering work that goes beyond fixing bugs and resolving issues.
SWE-Bench Pro
SWE-Bench Pro (stylized SWE-BENCH PRO) is a contamination-resistant benchmark, released by Scale AI in September 2025, that measures whether an AI coding agent can resolve long-horizon
SWE-Lancer
SWE-Lancer is a benchmark released by OpenAI in February 2025 that evaluates the ability of frontier large language models to perform real-world freelance software-engineering work.
SWE-bench
SWE-bench is an execution-based benchmark for evaluating whether a language-model system can resolve real software issues.
SWE-bench Multilingual
SWE-bench Multilingual is an AI benchmark of 300 real-world software bug-fixing tasks drawn from 42 open-source repositories across nine programming languages
SWE-bench Multimodal
SWE-bench Multimodal (also written SWE-bench M) is a benchmark that measures whether autonomous software-engineering systems can resolve bugs in visual
SWE-bench Verified
SWE-bench Verified is a 500-problem, human-validated subset of the SWE-bench software engineering benchmark, released on August 13
SWE-rebench
SWE-rebench is a continuously refreshed, contamination-resistant AI benchmark and public leaderboard for evaluating AI agents on real-world software engineering tasks.
SciCode
SciCode is a research coding benchmark that asks large language models to write Python code for realistic, PhD level scientific problems drawn from working scientists' day to day workflows.
ScienceAgentBench
ScienceAgentBench is an AI benchmark for evaluating whether language model agents can perform real, data-driven scientific analysis by writing and executing code.
SimpleBench
SimpleBench is a text-only benchmark for large language models created by Philip, the host of the AI Explained YouTube channel, with collaborator Hemang.
SimpleQA
SimpleQA is a factuality benchmark released by OpenAI on October 30, 2024 that measures whether large language models can answer short, fact-seeking questions correctly instead of producing hallucinations.
SimpleQA Verified
SimpleQA Verified is a short-form factuality benchmark released by Google DeepMind and Google Research in September 2025 that measures the parametric knowledge of large language models using roughly 1,000…
SkillsBench
SkillsBench is a benchmark for measuring whether Agent Skills, the structured packages of procedural knowledge that augment AI agents at inference time, actually improve how well those agents do real work.
Specific Labs
Specific Labs is a San Francisco company that buys and licenses operational data and source code from businesses, and packages it as training and evaluation material for AI labs.
Spider 2.0
Spider 2.0 is a benchmark for evaluating large language models on real-world enterprise text-to-SQL workflows.
StrongREJECT
StrongREJECT is a benchmark and automated evaluator for measuring how well jailbreaking attacks actually work against large language models.
SuperGLUE
SuperGLUE is a public benchmark for evaluating general-purpose English language understanding in machine learning systems, consisting of eight challenging reasoning and reading-comprehension tasks, two…
SuperGPQA
SuperGPQA is a large graduate-level knowledge and reasoning benchmark for evaluating large language models across 285 academic disciplines.
T2I-CompBench
T2I-CompBench is an AI benchmark for evaluating compositional text-to-image generation, introduced in the paper "T2I-CompBench: A Comprehensive Benchmark for Open-world Compositional Text-to-image Generation"…
Task-completion time horizon (METR)
The task-completion time horizon is a metric for AI capability proposed by METR that expresses a model's ability in units of human time: it is the length of task
Tau2-bench
τ²-bench (also written Tau2-bench or τ^2-bench) is a benchmark for evaluating conversational AI agents in dual-control environments, where both the agent and a simulated user can call tools to read from and…
Terminal-Bench
Terminal-Bench is an open benchmark for evaluating AI agents on complex, real-world tasks performed through command-line terminal interfaces.
TheAgentCompany
TheAgentCompany is an AI benchmark that evaluates AI agents on long-horizon, economically valuable knowledge work inside a self-hosted simulation of a small software company.
Tokens per second
Tokens per second (TPS) is a key performance metric for measuring the speed of large language model (LLM) inference. It quantifies how many tokens a model can generate or process in one second.
ToxiGen
ToxiGen is a large-scale, machine-generated dataset designed for adversarial and implicit hate speech detection.
TriviaQA
TriviaQA is a large-scale reading comprehension and question answering dataset of over 650,000 question-answer-evidence triples, introduced in 2017 by Mandar Joshi, Eunsol Choi, Daniel S. Weld
TruthfulQA
TruthfulQA is a benchmark designed to measure whether large language models (LLMs) generate truthful answers to questions.
Vending-Bench
Vending-Bench is a benchmark that measures the long-term coherence of large language model-based AI agents by tasking them with running a simulated vending-machine business over an extended horizon.
Vibe Code Bench
Vibe Code Bench (VCB) is a benchmark that measures whether AI models can build a complete, deployable web application from nothing but a written specification.
Video-MME
Video-MME (Video Multi-Modal Evaluation) is a benchmark for testing how well multimodal large language models (MLLMs) understand video, built from 900 manually selected videos totaling 254 hours and 2,700…
Video-MMMU
Video-MMMU (Video Multi-Modal Multi-disciplinary Understanding, sometimes written VideoMMMU) is a benchmark that measures whether Large Multimodal Models can acquire new knowledge from professional educational…
Vimgolf
vimgolf-gym is an OpenAI Gym style customizable environment and benchmark built around VimGolf, the long running keystroke counting puzzle game for Vim.
Visual Question Answering Models
Visual question answering models are AI systems that take an image and a natural language question about that image and return a natural language answer.
Visual question answering
Visual question answering (VQA) is the task of producing a natural-language answer to a natural-language question about an image.
VisualWebArena
VisualWebArena (often abbreviated VWA) is a benchmark of 910 realistic, visually grounded web tasks for evaluating multimodal autonomous AI agents, released in January 2024 by researchers at Carnegie Mellon…
WISE
WISE (World Knowledge-Informed Semantic Evaluation) is an AI benchmark that tests whether a text-to-image model actually possesses and correctly applies real-world knowledge when it draws a scene
WMDP benchmark
The Weapons of Mass Destruction Proxy (WMDP) is a publicly released benchmark and unlearning testbed for large language models, introduced in the paper "The WMDP Benchmark: Measuring and Reducing Malicious Use…
WebArena
WebArena is a realistic, self-hosted web environment and benchmark designed for developing and evaluating autonomous AI agents that perform tasks on the web.
WebDev Arena
WebDev Arena is a live, community-driven leaderboard that ranks large language models on their ability to generate working web applications.
WebVoyager
WebVoyager is an end-to-end web agent and its companion benchmark, introduced by Hongliang He and seven coauthors in a paper accepted to ACL 2024 .
WeirdML
WeirdML is a benchmark for evaluating how well large language models can do hands on machine learning engineering.
WildBench
WildBench is an automated evaluation framework for large language models (LLMs) introduced by the Allen Institute for AI (AI2) in 2024.
Windows Agent Arena
Windows Agent Arena (WAA) is a reproducible benchmark and evaluation environment created by Microsoft for testing multimodal computer-use agents on real Windows tasks at scale.
WindowsWorld
WindowsWorld is a process-aware benchmark for evaluating autonomous computer-use agents on professional workflows that cross several Windows applications.
WinoGrande
WinoGrande is a large-scale benchmark for commonsense reasoning consisting of 44,000 binary fill-in-the-blank pronoun resolution problems, built to test whether language models genuinely understand commonsense…
Winograd Schema Challenge
The Winograd Schema Challenge (WSC) is a commonsense reasoning test in which a system must resolve an ambiguous pronoun in a short sentence where the correct answer flips when one or two words change
WritingBench
WritingBench is a comprehensive benchmark for evaluating the generative writing capabilities of large language models (LLMs) across diverse real-world writing tasks.
ZebraLogic
ZebraLogic is a benchmark for evaluating the logical reasoning capabilities of large language models (LLMs).
ZeroBench
ZeroBench is a visual reasoning benchmark built to be effectively impossible for current frontier large multimodal models, which score 0.0% on its main questions.
τ-bench
τ-bench (Tau-bench), short for Tool-Agent-User Interaction Benchmark, is an AI benchmark that evaluates language agents' ability to complete complex tasks through realistic, multi-turn interactions with…