LingoQA
LingoQA is a video question answering benchmark and dataset for autonomous driving, developed by researchers at the London-based self-driving company Wayve and published as the paper "LingoQA: Visual Question…
AI Benchmarks
Explore AI Benchmarks through related topics and the articles other pages reference most.
Explore articles
Browse subtopics (51)
Articles that also belong to these categories. Counts cover all of AI Benchmarks.
All articles (239)Model Evaluation (56)Natural Language Processing (42)Large Language Models (33)AI Agents (28)AI Code Generation (25)AI Safety (25)Machine Learning (22)Computer Vision (19)Multimodal AI (15)Reasoning Models (13)Data & Datasets (11)Artificial Intelligence (8)OpenAI (6)AI Research (5)Healthcare AI (4)Mathematics (4)Open Source AI (4)AI Companies (3)AI Ethics (3)AI Hardware (3)Developer Tools (3)Google DeepMind (3)Image Generation (3)Information Retrieval (3)NVIDIA (3)Robotics (3)Reinforcement Learning (2)Software Development (2)Speech & Audio AI (2)Statistics (2)World Models (2)2025 in artificial intelligence (1)AI Alignment (1)AI Inference (1)AI Models (1)AI in Cybersecurity (1)Autonomous Vehicles (1)Data Science (1)Deep Learning (1)Education AI (1)Embodied AI (1)Enterprise AI (1)Finance AI (1)Generative AI (1)Legal AI (1)Meta AI (1)Microsoft (1)People (1)Physical AI (1)Research Organizations (1)Video Generation (1)
Showing 121-180 of 239 articles
LiveBench
LiveBench is a contamination-limited benchmark for large language models (LLMs) that draws frequently updated questions from recent sources (math competitions, arXiv papers, news articles, and datasets) and…
LiveCodeBench
LiveCodeBench is a holistic and contamination-free benchmark for evaluating large language models on code, first released in March 2024 by researchers at UC Berkeley, MIT, and Cornell led by Naman Jain.
LongBench
LongBench is a benchmark suite for evaluating the long-context understanding capabilities of large language models (LLMs).
LongBench v2
LongBench v2 is a benchmark for evaluating how well large language models understand and reason over long contexts.
LongFact / SAFE
LongFact and SAFE are a paired benchmark and evaluation method for measuring the long-form factuality of large language models, introduced by researchers at Google DeepMind and Stanford University in the 2024…
Longform Creative Writing
Longform Creative Writing is a capability area and benchmark family for large language models that measures how well a system can write coherent, engaging fiction or narrative non-fiction over thousands of…
MACHIAVELLI (benchmark)
MACHIAVELLI is a benchmark for evaluating the ethical behavior of AI agents in text-based interactive environments.
MASK
MASK (Model Alignment between Statements and Knowledge) is an AI safety benchmark that measures the honesty of large language models (LLMs) by testing whether a model will knowingly assert something it…
MATH
MATH is a benchmark of 12,500 competition mathematics problems used to evaluate the mathematical problem-solving ability of machine learning systems, particularly large language models.
MATH Level 5
MATH Level 5 is the hardest difficulty tier of the MATH (benchmark) dataset introduced by Dan Hendrycks and colleagues in the 2021 paper Measuring Mathematical Problem Solving With the MATH Dataset.
MATH-500
MATH-500 is a 500-problem benchmark for evaluating the mathematical reasoning of large language models, formed by holding out 500 problems from the test split of the MATH benchmark of Dan Hendrycks et al.
MBPP
MBPP (Mostly Basic Python Problems) is a code generation benchmark of 974 crowd-sourced Python programming tasks designed to be solvable by entry-level programmers, introduced by Jacob Austin, Augustus Odena…
METR
METR (Model Evaluation and Threat Research) is a nonprofit research organization based in Berkeley, California, that develops scientific methods for measuring the autonomous capabilities of frontier AI systems…
MGSM (Multilingual Grade School Math)
MGSM (Multilingual Grade School Math) is a multilingual benchmark that measures the mathematical reasoning ability of large language models by taking 250 grade-school math word problems from GSM8K and…
MLE-bench
MLE-bench is a benchmark for measuring how well autonomous AI agents perform real-world machine learning engineering, released by OpenAI's Preparedness team on October 9, 2024 (arXiv:2410.07095).
MLPerf
MLPerf is the AI industry's standard suite of benchmarks for measuring how fast machine learning systems train and run models, developed and maintained by MLCommons
MM-BrowseComp
MM-BrowseComp is a benchmark for evaluating multimodal web-browsing AI agents, introduced in August 2025 by researchers from ByteDance, Nanjing University, M-A-P, the Institute of Automation of the Chinese…
MM-Vet
MM-Vet is an AI benchmark for evaluating large multimodal models (LMMs, also called multimodal large language models or MLLMs) on tasks that require combining several core vision-language skills at once.
MMLU
MMLU, short for Massive Multitask Language Understanding, is an English-language AI benchmark for evaluating broad academic and professional knowledge through four-option multiple-choice questions.
MMLU-Pro
MMLU-Pro (Massive Multitask Language Understanding Professional) is an artificial intelligence benchmark of 12,032 ten-choice questions across 14 academic domains
MMLU-ProX
MMLU-ProX is a multilingual benchmark for evaluating reasoning and knowledge in large language models
MMLU-Redux
MMLU-Redux is a manually re-annotated, error-corrected subset of the Massive Multitask Language Understanding (MMLU) benchmark
MMMLU
MMMLU (Multilingual Massive Multitask Language Understanding) is a multilingual evaluation dataset published by OpenAI on September 23, 2024.
MMMU
MMMU (Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark) is a multimodal AI benchmark of 11,550 college-level questions that pairs text with images to test expert knowledge and…
MMMU-Pro
MMMU-Pro is a rigorous benchmark for evaluating multimodal AI systems on college-level, expert questions that genuinely require seeing an image, built as a harder and more robust version of the original MMMU…
MMStar
MMStar (Multi-modal Star) is a vision-language model evaluation benchmark consisting of 1,500 multimodal samples that were filtered from six pre-existing benchmarks to ensure both visual dependency (questions…
MMTEB
MMTEB (Massive Multilingual Text Embedding Benchmark) is a large, community-built suite for evaluating text embedding models across more than 500 quality-controlled tasks and over 250 languages
MRCR
MRCR (Multi-Round Co-reference Resolution) is a synthetic long-context evaluation that tests whether a large language model can locate and disambiguate among several near-identical "needles" buried inside a…
MT-Bench
MT-Bench (Multi-Turn Benchmark) is a benchmark of 80 hand-written, two-turn questions that evaluates large language models (LLMs) on multi-turn conversation and instruction following by using a strong model…
MTEB (Massive Text Embedding Benchmark)
MTEB, short for Massive Text Embedding Benchmark, is the standard public leaderboard for evaluating text embedding models across many task types at once.
MathArena
MathArena is a public, continuously updated leaderboard and evaluation platform that measures the performance of large language models on mathematics competition problems released after each model's training…
MathVista
MathVista is a benchmark for evaluating the mathematical reasoning capabilities of foundation models in visual contexts.
Mathematical reasoning in AI
Mathematical reasoning in AI is the ability of computer systems to solve mathematical problems: carrying out multi-step calculations, proving theorems, and answering competition or research questions that…
MedHELM
MedHELM (Holistic Evaluation of Large Language Models for Medical Tasks) is a benchmark and evaluation framework that measures how well large language models perform on realistic clinical work.
MedQA
MedQA is a large-scale, open-domain medical question answering benchmark of multiple-choice questions taken from real medical licensing examinations, introduced by Di Jin and colleagues at MIT in 2020.
Mike Knoop
Mike Knoop is an American technology entrepreneur and artificial intelligence researcher who co-founded the workflow-automation company Zapier in 2011 and, in 2024, co-founded the ARC Prize
Mind2Web
Mind2Web is the first large-scale dataset and benchmark for building and evaluating generalist AI agents that follow natural language instructions to complete tasks on real websites.
MuSR
MuSR (Multistep Soft Reasoning) is a benchmark for evaluating multistep reasoning in large language models, built around long free-text narratives such as murder mysteries, object-placement scenarios, and…
Multi-SWE-bench
Multi-SWE-bench is a multilingual benchmark for evaluating the ability of large language model based coding systems to resolve real-world software issues across seven programming languages.
MultiChallenge
MultiChallenge is an AI benchmark for evaluating large language models on realistic multi-turn conversations.
NVIDIA Isaac Lab-Arena
NVIDIA Isaac Lab-Arena is an open-source framework for composing simulated robot tasks and evaluating learned policies at scale.
NVIDIA Nemotron Model Reasoning Challenge
The NVIDIA Nemotron Model Reasoning Challenge was a Kaggle competition run by NVIDIA from March to June 2026 in which participants tried to improve the reasoning accuracy of a fixed open model, Nemotron 3 Nano…
Natural language inference (NLI)
Natural language inference (NLI), also known as recognising textual entailment (RTE), is the natural language processing task of deciding whether a hypothesis sentence is entailed by, contradicts, or is…
Needle in a Haystack (NIAH)
Needle in a Haystack (NIAH) is a long-context evaluation that measures whether a large language model can retrieve a single fact (the "needle") inserted at a controlled position inside a long body of text (the…
NoLiMa
NoLiMa, short for "No Literal Matching," is a long-context benchmark for large language models that measures how well a model can find and use a single relevant fact buried in a long document when that fact…
OSWorld
OSWorld is a benchmark for evaluating multimodal AI agents on open-ended tasks performed inside real computer environments.
OlympiadBench
OlympiadBench is an AI benchmark of 8,476 Olympiad-level mathematics and physics problems, designed to test the advanced scientific reasoning of large language models and large multimodal models.
Omni-MATH
Omni-MATH is an AI benchmark of Olympiad-level competition mathematics, introduced in October 2024 to measure the mathematical reasoning ability of large language models on problems far harder than those in…
Online-Mind2Web
Online-Mind2Web is a live-web benchmark for measuring whether browser-based AI agents can complete multi-step tasks on real websites.
OpenAI MRCR (Multi-Round Co-reference Resolution)
OpenAI MRCR (Multi-Round Co-reference Resolution) is a long-context evaluation dataset published by OpenAI that measures a language model's ability to distinguish between multiple near-identical "needles"…
PASCAL VOC
PASCAL VOC (Pattern Analysis, Statistical Modelling and Computational Learning Visual Object Classes) is a long-running benchmark dataset and annual challenge for object recognition, object detection…
PIQA
PIQA (Physical Interaction Question Answering) is a benchmark dataset of roughly 21,000 binary multiple-choice questions that evaluates the physical commonsense reasoning abilities of natural language…
Paper2Video
Paper2Video (full title: Paper2Video: Automatic Video Generation from Scientific Papers) is a research project from Show Lab at the National University of Singapore that formalizes and evaluates automatic…
PaperBench
PaperBench is a benchmark released by OpenAI in April 2025 that measures whether AI agents can replicate cutting-edge machine learning research papers from scratch.
Pass@k
Pass@k is the standard metric for evaluating code generation models: it measures the probability that at least one of k generated candidate solutions passes all of a problem's unit tests.
ProcessBench
ProcessBench is a benchmark for step-level verification of mathematical reasoning, built by the Qwen Team at Alibaba and released in December 2024.
PubMedQA
PubMedQA is a biomedical question answering dataset and benchmark that evaluates whether machine learning models can answer yes/no/maybe research questions using evidence from PubMed abstracts.
PutnamBench
PutnamBench is a benchmark for evaluating automated theorem proving systems, especially neural and large-language-model-based provers, on competition mathematics.
RE-Bench
RE-Bench (short for Research Engineering Benchmark) is a benchmark for evaluating the frontier AI research-and-development capabilities of large language model agents, developed by the nonprofit METR (Model…