AA-LCR
AA-LCR (Artificial Analysis Long Context Reasoning) is a benchmark for large language models that evaluates the ability to reason across multiple real-world documents totalling approximately 100,000 tokens per…
AI Benchmarks
Explore AI Benchmarks through related topics and the articles other pages reference most.
Explore articles
Browse subtopics: Natural Language Processing
Articles that also belong to these categories. Counts cover all of AI Benchmarks.
All articles (239)Model Evaluation (56)Natural Language Processing (42)Large Language Models (33)AI Agents (28)AI Code Generation (25)AI Safety (25)Machine Learning (22)Computer Vision (19)Multimodal AI (15)Reasoning Models (13)Data & Datasets (11)Artificial Intelligence (8)OpenAI (6)AI Research (5)Healthcare AI (4)Mathematics (4)Open Source AI (4)AI Companies (3)AI Ethics (3)AI Hardware (3)Developer Tools (3)Google DeepMind (3)Image Generation (3)Information Retrieval (3)NVIDIA (3)Robotics (3)Reinforcement Learning (2)Software Development (2)Speech & Audio AI (2)Statistics (2)World Models (2)2025 in artificial intelligence (1)AI Alignment (1)AI Inference (1)AI Models (1)AI in Cybersecurity (1)Autonomous Vehicles (1)Data Science (1)Deep Learning (1)Education AI (1)Embodied AI (1)Enterprise AI (1)Finance AI (1)Generative AI (1)Legal AI (1)Meta AI (1)Microsoft (1)People (1)Physical AI (1)Research Organizations (1)Video Generation (1)
Showing 1-42 of 42 articles
AlpacaEval
AlpacaEval is an automatic evaluation framework for instruction-following large language models (LLMs) developed by Stanford University's Tatsu Lab
BBQ (Bias Benchmark for QA)
BBQ (the Bias Benchmark for QA) is a hand-built evaluation dataset that measures whether a question answering (QA) language model relies on social stereotypes when it answers.
BIG-Bench Hard
BIG-Bench Hard (BBH) is a suite of 23 challenging tasks drawn from the BIG-Bench benchmark, selected because they are "the [tasks] for which prior language model evaluations did not outperform the average…
BoolQ
BoolQ (Boolean Questions) is a natural language processing benchmark dataset of 15,942 naturally occurring yes/no question answering examples, each pairing a real Google search query with a Wikipedia passage…
CRUXEval
CRUXEval (Code Reasoning, Understanding, and eXecution Evaluation) is a benchmark designed to measure how well large language models can reason about, understand, and mentally execute short Python programs.
CommonsenseQA
CommonsenseQA is a multiple-choice question answering benchmark of 12,247 questions, introduced in 2019 by Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant
Coreference Resolution
Coreference resolution is the natural language processing task of finding all the expressions in a text that refer to the same entity and grouping them together.
DCLM (DataComp for Language Models)
DCLM, short for DataComp for Language Models (also styled DataComp-LM), is an open benchmark, dataset, and software framework, released in June 2024
DROP (Discrete Reasoning Over Paragraphs)
DROP** (Discrete Reasoning Over Paragraphs) is a reading comprehension benchmark that requires artificial intelligence systems to perform discrete reasoning operations over textual content.
FLORES-200
FLORES-200 is a multilingual evaluation benchmark for machine translation systems, covering 200 languages across a wide range of language families, scripts, and resource levels.
GLUE benchmark
The General Language Understanding Evaluation (GLUE) benchmark is a collection of nine natural language understanding (NLU) tasks designed to evaluate and compare the performance of language models across a…
Global-MMLU
Global-MMLU is a multilingual evaluation benchmark that extends the MMLU question-answering dataset across 42 languages, with designated subsets labeled culturally sensitive (CS) and culturally agnostic (CA).
HellaSwag
HellaSwag is a commonsense reasoning benchmark for language models, introduced by Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi in the 2019 paper "HellaSwag: Can a Machine Really…
HotpotQA
HotpotQA is a large-scale, multi-hop question answering dataset of about 112,779 crowd-authored question-and-answer pairs over English Wikipedia, whose answers cannot be found in any single paragraph and…
IFEval
IFEval (Instruction-Following Evaluation) is a benchmark of 541 prompts that measures how reliably large language models obey explicit, machine-checkable instructions such as "write in more than 400 words,"…
InfiniteBench
InfiniteBench (stylized as ∞Bench) is a long-context benchmark that tests whether large language models (LLMs) can genuinely process and reason over inputs longer than 100,000 tokens, using 12 tasks that span…
LAMBADA
LAMBADA (LAnguage Modeling Broadened to Account for Discourse Aspects) is a benchmark dataset designed to evaluate the ability of computational language models to understand broad discourse context.
LegalBench
LegalBench is a collaboratively constructed benchmark for measuring legal reasoning in large language models (LLMs)
LibriSpeech
LibriSpeech is a freely available corpus of approximately 1,000 hours of 16 kHz read English speech that serves as the standard benchmark for training and evaluating automatic speech recognition (ASR) systems.
LiveBench
LiveBench is a contamination-limited benchmark for large language models (LLMs) that draws frequently updated questions from recent sources (math competitions, arXiv papers, news articles, and datasets) and…
LongBench
LongBench is a benchmark suite for evaluating the long-context understanding capabilities of large language models (LLMs).
MGSM (Multilingual Grade School Math)
MGSM (Multilingual Grade School Math) is a multilingual benchmark that measures the mathematical reasoning ability of large language models by taking 250 grade-school math word problems from GSM8K and…
MMTEB
MMTEB (Massive Multilingual Text Embedding Benchmark) is a large, community-built suite for evaluating text embedding models across more than 500 quality-controlled tasks and over 250 languages
MT-Bench
MT-Bench (Multi-Turn Benchmark) is a benchmark of 80 hand-written, two-turn questions that evaluates large language models (LLMs) on multi-turn conversation and instruction following by using a strong model…
MTEB (Massive Text Embedding Benchmark)
MTEB, short for Massive Text Embedding Benchmark, is the standard public leaderboard for evaluating text embedding models across many task types at once.
MathArena
MathArena is a public, continuously updated leaderboard and evaluation platform that measures the performance of large language models on mathematics competition problems released after each model's training…
MedQA
MedQA is a large-scale, open-domain medical question answering benchmark of multiple-choice questions taken from real medical licensing examinations, introduced by Di Jin and colleagues at MIT in 2020.
Natural language inference (NLI)
Natural language inference (NLI), also known as recognising textual entailment (RTE), is the natural language processing task of deciding whether a hypothesis sentence is entailed by, contradicts, or is…
PIQA
PIQA (Physical Interaction Question Answering) is a benchmark dataset of roughly 21,000 binary multiple-choice questions that evaluates the physical commonsense reasoning abilities of natural language…
PubMedQA
PubMedQA is a biomedical question answering dataset and benchmark that evaluates whether machine learning models can answer yes/no/maybe research questions using evidence from PubMed abstracts.
SQuAD
SQuAD (the Stanford Question Answering Dataset) is a large-scale reading comprehension benchmark from Stanford University in which a model must answer a question by extracting the exact span of text that…
SimpleBench
SimpleBench is a text-only benchmark for large language models created by Philip, the host of the AI Explained YouTube channel, with collaborator Hemang.
SimpleQA
SimpleQA is a factuality benchmark released by OpenAI on October 30, 2024 that measures whether large language models can answer short, fact-seeking questions correctly instead of producing hallucinations.
SuperGLUE
SuperGLUE is a public benchmark for evaluating general-purpose English language understanding in machine learning systems, consisting of eight challenging reasoning and reading-comprehension tasks, two…
ToxiGen
ToxiGen is a large-scale, machine-generated dataset designed for adversarial and implicit hate speech detection.
TriviaQA
TriviaQA is a large-scale reading comprehension and question answering dataset of over 650,000 question-answer-evidence triples, introduced in 2017 by Mandar Joshi, Eunsol Choi, Daniel S. Weld
TruthfulQA
TruthfulQA is a benchmark designed to measure whether large language models (LLMs) generate truthful answers to questions.
WinoGrande
WinoGrande is a large-scale benchmark for commonsense reasoning consisting of 44,000 binary fill-in-the-blank pronoun resolution problems, built to test whether language models genuinely understand commonsense…
Winograd Schema Challenge
The Winograd Schema Challenge (WSC) is a commonsense reasoning test in which a system must resolve an ambiguous pronoun in a short sentence where the correct answer flips when one or two words change
WritingBench
WritingBench is a comprehensive benchmark for evaluating the generative writing capabilities of large language models (LLMs) across diverse real-world writing tasks.
ZebraLogic
ZebraLogic is a benchmark for evaluating the logical reasoning capabilities of large language models (LLMs).