Showing 61-120 of 220 articles
Factorio Learning Environment
Factorio Learning Environment Abbreviation An open-ended evaluation framework that uses the industrial automation game Factorio to test long-horizon...
FeatureBench
FeatureBench is an execution-based benchmark for measuring how well LLM-powered coding agents handle complex, feature-oriented software development rather than...
AI Code Generation
FinanceBench
FinanceBench is an AI benchmark for open-book financial question answering, designed to test whether large language models can answer the kinds of questions a...
AI Safety
FormulaOne
FormulaOne is an AI benchmark designed to measure the depth of algorithmic reasoning in frontier large language models, introduced in the July 2025 paper...
Model Evaluation
Fox (benchmark)
Fox is an evaluation suite for fine-grained, multi-page document understanding by large vision-language models. It was released in May 2024 alongside a model...
Computer VisionMultimodal AI
Frechet Inception Distance
The Frechet Inception Distance (FID) is the standard metric for measuring the quality of images produced by generative models: it computes the Frechet distance...
Computer VisionGenerative AI
FrontierMath
FrontierMath: A Benchmark for Evaluating Advanced Mathematical Reasoning in AI Abbreviation A benchmark of research-level mathematics problems designed to...
Artificial IntelligenceMathematics
GAIA benchmark
GAIA (General AI Assistants) is a benchmark for evaluating general-purpose AI agents and assistants on real-world tasks that require reasoning, web browsing,...
AI Agents
GDPval
GDPval is a benchmark released by OpenAI on September 25, 2025 that measures how well frontier AI models can produce the actual deliverables of professional...
AI Research
GLUE benchmark
The General Language Understanding Evaluation (GLUE) benchmark is a collection of nine natural language understanding (NLU) tasks designed to evaluate and...
Machine LearningNatural Language Processing
GPQA
GPQA (Graduate-Level Google-Proof Q&A) is a benchmark of 448 expert-written multiple-choice questions in biology, physics, and chemistry, designed to test...
Reasoning Models
GPQA Diamond
GPQA Diamond is the 198-question hardest subset of the Graduate-Level Google-Proof Q&A Benchmark (GPQA), a set of PhD-level multiple-choice questions in...
GSM8K
GSM8K (Grade School Math 8K) is a benchmark dataset of 8,792 grade-school-level math word problems created by researchers at OpenAI to evaluate the multi-step...
Large Language ModelsMachine Learning
GSO
GSO (short for Global Software Optimization in the project page subtitle, and Challenging Software Optimization Tasks for Evaluating SWE-Agents in the paper...
GSO (Global Software Optimization Bench)
GSO (Global Software Optimization), also styled as a benchmark of "Challenging Software Optimization Tasks for Evaluating SWE-Agents," is an AI benchmark that...
AI Code Generation
GenAI-Bench
GenAI-Bench is an AI benchmark for evaluating compositional text-to-image and text-to-video generation, introduced in 2024 by researchers from Carnegie Mellon...
Computer Vision
GenEval
GenEval is an object-focused benchmark for evaluating how well text-to-image models follow the content of a prompt. Instead of scoring overall image quality,...
Computer VisionImage Generation
GeoBench
Geospatial Benchmarks Collection Description June 2023 (GEO-Bench v1) Latest variant Alexandre Lacoste et al. (GEO-Bench), Muhammad Sohail Danish et al....
Global-MMLU
Global-MMLU is a multilingual evaluation benchmark that extends the MMLU question-answering dataset across 42 languages, with designated subsets labeled...
Model EvaluationNatural Language Processing
HELM (Holistic Evaluation of Language Models)
HELM (Holistic Evaluation of Language Models) is an open-source benchmark framework created by the Center for Research on Foundation Models (CRFM) at Stanford...
Model Evaluation
HELMET
HELMET (How to Evaluate Long-context Models Effectively and Thoroughly) is a benchmark for evaluating long-context language models introduced by researchers at...
Model Evaluation
HalluLens
HalluLens is a large language model hallucination benchmark introduced by researchers at Meta AI's Fundamental AI Research (FAIR) lab, together with...
Meta AIModel Evaluation
HaluEval
HaluEval (Hallucination Evaluation) is a large-scale benchmark for measuring how well large language models (LLMs) can recognize hallucinated content, that is,...
AI SafetyLarge Language Models
HarmBench
HarmBench is a standardized evaluation framework for automated red teaming and robust refusal of large language models (LLMs). It was introduced in 2024 by...
AI SafetyLarge Language Models
Harness (AI)
This article is about software that wraps and evaluates AI models. For the 2015 paper "Explaining and Harnessing Adversarial Examples" and other uses of the...
AI AgentsModel Evaluation
HealthBench
--- Full name HealthBench Description May 12, 2025 (paper May 13, 2025) Latest version Rahul K. Arora, Jason Wei, Rebecca Soskin Hicks, Preston...
Healthcare AIOpenAI
HealthBench Hard
HealthBench Hard Abbreviation A 1,000 example curated subset of HealthBench, selected because frontier models score poorly on it Release date 1.0 ...
Healthcare AIOpenAI
HellaSwag
HellaSwag is a commonsense reasoning benchmark for language models, introduced by Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi in the...
Natural Language Processing
HotpotQA
HotpotQA is a large-scale, multi-hop question answering dataset of about 112,779 crowd-authored question-and-answer pairs over English Wikipedia, whose answers...
Artificial IntelligenceData & Datasets
HumanEval
HumanEval is a benchmark for evaluating the code generation capabilities of large language models, consisting of 164 hand-written Python programming problems,...
AI Code Generation
Humanity's Last Exam
Humanity's Last Exam Abbreviation Multi-modal AI benchmark testing frontier knowledge across 100+ academic subjects, designed to be the most challenging...
AI Safety
IFBench
IFBench (Instruction Following Benchmark) is an artificial intelligence benchmark that measures whether large language models can follow precise output...
IFEval
IFEval (Instruction-Following Evaluation) is a benchmark of 541 prompts that measures how reliably large language models obey explicit, machine-checkable...
Large Language ModelsNatural Language Processing
InfiniteBench
InfiniteBench (stylized as ∞Bench) is a long-context benchmark that tests whether large language models (LLMs) can genuinely process and reason over inputs...
Large Language ModelsNatural Language Processing
Iris dataset
The Iris dataset, sometimes referred to as Fisher's Iris dataset or the Iris flower dataset, is a multivariate dataset introduced by the British statistician...
Data & DatasetsMachine Learning
JailbreakBench
JailbreakBench is an open-source robustness benchmark for evaluating jailbreak attacks and defenses against large language models (LLMs). It bundles four...
AI SafetyLarge Language Models
KernelBench
KernelBench is an AI benchmark and open-source evaluation environment that measures how well large language models can write fast and correct GPU kernels. The...
Model Evaluation
LAB-Bench
LAB-Bench (the Language Agent Biology Benchmark) is an AI benchmark of more than 2,400 multiple-choice questions built to measure how well language models and...
Model Evaluation
LAMBADA
LAMBADA (LAnguage Modeling Broadened to Account for Discourse Aspects) is a benchmark dataset designed to evaluate the ability of computational language models...
Natural Language Processing
LIBERO
LIBERO ("LIfelong learning BEnchmark on RObot manipulation tasks") is an AI benchmark for studying knowledge transfer in lifelong robot learning. Introduced in...
Robotics
LLM Benchmark Comparison (Leaderboard Overview)
As of July 2026, no single model wins every LLM benchmark, but Anthropic's Claude Fable 5 tops the most: it currently leads MMLU-Pro (91.5%), SWE-bench...
Large Language ModelsModel Evaluation
LLM Benchmarks Timeline
The LLM benchmarks timeline is the chronological record of the major tests used to evaluate large language models, from the BERT era reading-comprehension...
LLM Comparisons
LLM comparison is the practice of ranking large language models against one another on quality, cost, speed, and specific skills, using public leaderboards,...
LLM Rankings
LLM rankings order large language models from best to worst using one of two broad methods: human preference voting and fixed benchmark scoring. The most-cited...
LLM-as-a-judge
LLM-as-a-judge is the practice of using a strong large language model to evaluate the outputs of other models, or of itself, in place of a human annotator....
Large Language ModelsModel Evaluation
LM Evaluation Harness
LM Evaluation Harness (the Language Model Evaluation Harness, often abbreviated lm-eval or written lm-evaluation-harness) is an open-source software framework...
Model EvaluationOpen Source AI
LMArena
LMArena is a crowdsourced artificial intelligence evaluation platform and company that ranks large language models by having anonymous users vote on which of...
AI CompaniesModel Evaluation
LegalBench
LegalBench is a collaboratively constructed benchmark for measuring legal reasoning in large language models (LLMs), consisting of 162 tasks that span six...
Large Language ModelsLegal AI
LibriSpeech
LibriSpeech is a freely available corpus of approximately 1,000 hours of 16 kHz read English speech that serves as the standard benchmark for training and...
Natural Language ProcessingSpeech & Audio AI
LiveBench
LiveBench Description 2024-06-12 Latest version 2026-01-08 Authors Abacus.AI, NYU, NVIDIA, University of Maryland, USC Technical Details Type Text...
Machine LearningNatural Language Processing
LiveCodeBench
LiveCodeBench is a holistic and contamination-free benchmark for evaluating large language models on code, first released in March 2024 by researchers at UC...
AI Code GenerationMachine Learning
LongBench
LongBench is a benchmark suite for evaluating the long-context understanding capabilities of large language models (LLMs). Developed by researchers at Tsinghua...
Large Language ModelsNatural Language Processing
LongBench v2
LongBench v2 is a benchmark for evaluating how well large language models understand and reason over long contexts. It consists of 503 challenging...
Large Language ModelsModel Evaluation
LongFact / SAFE
LongFact and SAFE are a paired benchmark and evaluation method for measuring the long-form factuality of large language models, introduced by researchers at...
AI Safety
Longform Creative Writing
Longform Creative Writing is a capability area and benchmark family for large language models that measures how well a system can write coherent, engaging...
MACHIAVELLI (benchmark)
MACHIAVELLI is a benchmark for evaluating the ethical behavior of AI agents in text-based interactive environments. Introduced in 2023 by Alexander Pan, Jun...
AI AlignmentAI Ethics
MASK
MASK (Model Alignment between Statements and Knowledge) is an AI safety benchmark that measures the honesty of large language models (LLMs) by testing whether...
AI Safety
MATH
MATH is a benchmark of 12,500 competition mathematics problems used to evaluate the mathematical problem-solving ability of machine learning systems,...
Model EvaluationReasoning Models
MATH Level 5
MATH dataset, Level 5 difficulty subset Abbreviation The hardest difficulty tier of the MATH (benchmark) dataset, comprising competition mathematics...
MATH-500
MATH-500 is a 500-problem benchmark for evaluating the mathematical reasoning of large language models, formed by holding out 500 problems from the test split...
Mathematics