ARC-AGI-2
ARC-AGI-2 (Abstraction and Reasoning Corpus for Artificial General Intelligence 2) is an abstract reasoning benchmark for artificial intelligence, released on March 24, 2025 by the ARC Prize Foundation
AI Benchmarks
Explore AI Benchmarks through related topics and the articles other pages reference most.
Explore articles
Browse subtopics: Machine Learning
Articles that also belong to these categories. Counts cover all of AI Benchmarks.
All articles (239)Model Evaluation (56)Natural Language Processing (42)Large Language Models (33)AI Agents (28)AI Code Generation (25)AI Safety (25)Machine Learning (22)Computer Vision (19)Multimodal AI (15)Reasoning Models (13)Data & Datasets (11)Artificial Intelligence (8)OpenAI (6)AI Research (5)Healthcare AI (4)Mathematics (4)Open Source AI (4)AI Companies (3)AI Ethics (3)AI Hardware (3)Developer Tools (3)Google DeepMind (3)Image Generation (3)Information Retrieval (3)NVIDIA (3)Robotics (3)Reinforcement Learning (2)Software Development (2)Speech & Audio AI (2)Statistics (2)World Models (2)2025 in artificial intelligence (1)AI Alignment (1)AI Inference (1)AI Models (1)AI in Cybersecurity (1)Autonomous Vehicles (1)Data Science (1)Deep Learning (1)Education AI (1)Embodied AI (1)Enterprise AI (1)Finance AI (1)Generative AI (1)Legal AI (1)Meta AI (1)Microsoft (1)People (1)Physical AI (1)Research Organizations (1)Video Generation (1)
Showing 1-22 of 22 articles
BIG-Bench
BIG-Bench (Beyond the Imitation Game Benchmark) is a large-scale, collaborative benchmark of 204 tasks, contributed by 450 authors across 132 institutions, built to measure and extrapolate the capabilities of…
BIG-Bench Hard
BIG-Bench Hard (BBH) is a suite of 23 challenging tasks drawn from the BIG-Bench benchmark, selected because they are "the [tasks] for which prior language model evaluations did not outperform the average…
CRUXEval
CRUXEval (Code Reasoning, Understanding, and eXecution Evaluation) is a benchmark designed to measure how well large language models can reason about, understand, and mentally execute short Python programs.
CodeContests
CodeContests is a competitive programming dataset created by Google DeepMind for training and evaluating machine learning models on algorithmic problem-solving tasks.
Coreference Resolution
Coreference resolution is the natural language processing task of finding all the expressions in a text that refer to the same entity and grouping them together.
DROP (Discrete Reasoning Over Paragraphs)
DROP** (Discrete Reasoning Over Paragraphs) is a reading comprehension benchmark that requires artificial intelligence systems to perform discrete reasoning operations over textual content.
Elo rating system (AI model ranking)
The Elo rating system, as applied to AI models, is a method for turning a pile of head-to-head preference votes into a single number per model, so that large language models and chatbots can be ranked on a…
GLUE benchmark
The General Language Understanding Evaluation (GLUE) benchmark is a collection of nine natural language understanding (NLU) tasks designed to evaluate and compare the performance of language models across a…
GSM8K
GSM8K (Grade School Math 8K) is an English-language benchmark of grade-school arithmetic word problems released by OpenAI researchers in 2021.
Image classification
Image classification is the task of assigning a whole image to one category drawn from a fixed set of labels.
Iris dataset
The Iris dataset, sometimes referred to as Fisher's Iris dataset or the Iris flower dataset, is a multivariate dataset introduced by the British statistician and biologist Ronald Fisher in his 1936 paper "The…
LiveBench
LiveBench is a contamination-limited benchmark for large language models (LLMs) that draws frequently updated questions from recent sources (math competitions, arXiv papers, news articles, and datasets) and…
LiveCodeBench
LiveCodeBench is a holistic and contamination-free benchmark for evaluating large language models on code, first released in March 2024 by researchers at UC Berkeley, MIT, and Cornell led by Naman Jain.
MBPP
MBPP (Mostly Basic Python Problems) is a code generation benchmark of 974 crowd-sourced Python programming tasks designed to be solvable by entry-level programmers, introduced by Jacob Austin, Augustus Odena…
MLPerf
MLPerf is the AI industry's standard suite of benchmarks for measuring how fast machine learning systems train and run models, developed and maintained by MLCommons
MMLU-Pro
MMLU-Pro (Massive Multitask Language Understanding Professional) is an artificial intelligence benchmark of 12,032 ten-choice questions across 14 academic domains
MMMU
MMMU (Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark) is a multimodal AI benchmark of 11,550 college-level questions that pairs text with images to test expert knowledge and…
PASCAL VOC
PASCAL VOC (Pattern Analysis, Statistical Modelling and Computational Learning Visual Object Classes) is a long-running benchmark dataset and annual challenge for object recognition, object detection…
Pass@k
Pass@k is the standard metric for evaluating code generation models: it measures the probability that at least one of k generated candidate solutions passes all of a problem's unit tests.
SUPERB
SUPERB, which stands for Speech processing Universal PERformance Benchmark, is a comprehensive evaluation framework designed to measure how well self-supervised learning (SSL) models generalize across a…
TruthfulQA
TruthfulQA is a benchmark designed to measure whether large language models (LLMs) generate truthful answers to questions.