Model Evaluation

Explore Model Evaluation through related topics and the articles other pages reference most.

134 articlesSearch and filter Follow updates by RSS

Explore articles

Browse subtopics: AI Benchmarks

Articles that also belong to these categories. Counts cover all of Model Evaluation.

Showing 1-56 of 56 articles

AGIEval

AGIEval is an AI benchmark for evaluating foundation models on tasks that were originally designed for, and taken by, humans.

AI Benchmarks

ARC-AGI-2 (Abstraction and Reasoning Corpus for Artificial General Intelligence 2) is an abstract reasoning benchmark for artificial intelligence, released on March 24, 2025 by the ARC Prize Foundation

2025 in artificial intelligence AI Benchmarks

Agent evaluation

Agent evaluation is the systematic measurement of how well AI agents (LLM-based systems that plan and act over multiple steps using tools) perform on real-world tasks, using benchmarks, metrics, and testing…

AI Agents AI Benchmarks

Arena-Hard

Arena-Hard (and its evaluation tool Arena-Hard-Auto) is an automatic large language model (LLM) benchmark developed by the team behind Chatbot Arena that scores instruction-tuned models on 500 challenging

AI Benchmarks

BABILong

BABILong is a benchmark for testing how well a large language model can reason over facts scattered through very long text.

AI Benchmarks Large Language Models

Benchmark (AI)

In artificial intelligence and machine learning, a benchmark is a specified evaluation used to compare systems under common conditions.

AI Benchmarks

BigFinanceBench

BigFinanceBench (BFB) is a benchmark for evaluating how well AI agents perform the research work of professional financial analysts, developed by the finance-AI company Rogo and announced on May 27, 2026.

AI Benchmarks Finance AI

CaliBench

CaliBench is a benchmark for image-to-video generative models that asks whether a model reproduces the correct distribution of physical outcomes across many generations from the same starting frame

AI Benchmarks Video Generation

ChemBench

ChemBench is an automated AI benchmark that measures the chemical knowledge, reasoning, and safety judgment of large language models and compares their performance against expert human chemists.

AI Benchmarks

Coreference Resolution

Coreference resolution is the natural language processing task of finding all the expressions in a text that refer to the same entity and grouping them together.

AI Benchmarks Machine Learning

Cybench

Cybench (short for Cybersecurity benchmark) is an open-source evaluation framework for measuring the cybersecurity capabilities and risks of large language model agents.

AI Benchmarks AI Safety

Data contamination

Data contamination is the presence of benchmark evaluation data in the corpus used to train a model

AI Benchmarks Data & Datasets

DesignArena

DesignArena (also written Design Arena) is a crowdsourced benchmark and consumer creation platform for AI-generated design, built by the San Francisco startup Intelligence.

AI Benchmarks AI Companies

Elo rating system (AI model ranking)

The Elo rating system, as applied to AI models, is a method for turning a pile of head-to-head preference votes into a single number per model, so that large language models and chatbots can be ranked on a…

AI Benchmarks Machine Learning

EnigmaEval

EnigmaEval is an AI benchmark of long, complex multimodal puzzles drawn from real-world puzzle hunts, designed to measure the unstructured, creative, multi-step reasoning abilities of frontier AI models.

AI Benchmarks

FACTS Grounding

FACTS Grounding is a factuality benchmark from Google DeepMind and Google Research that measures whether a large language model answers a request using only the information in a provided source document

AI Benchmarks Large Language Models

FRAMES (benchmark)

FRAMES is an evaluation dataset for retrieval-augmented generation that tests factual accuracy, retrieval, and reasoning together rather than one at a time.

AI Benchmarks Information Retrieval

FormulaOne

FormulaOne is an AI benchmark for evaluating whether a model can design and implement dynamic programming algorithms for graph problems.

AI Benchmarks

Global-MMLU

Global-MMLU is a multilingual evaluation benchmark that extends the MMLU question-answering dataset across 42 languages, with designated subsets labeled culturally sensitive (CS) and culturally agnostic (CA).

AI Benchmarks Natural Language Processing

HELM (Holistic Evaluation of Language Models)

HELM (Holistic Evaluation of Language Models) is an open-source benchmark framework created by the Center for Research on Foundation Models (CRFM) at Stanford University for the reproducible and transparent…

AI Benchmarks

HELMET

HELMET (How to Evaluate Long-context Models Effectively and Thoroughly) is a benchmark for evaluating long-context language models introduced by researchers at Princeton University and Intel Labs in 2024.

AI Benchmarks

HalluLens

HalluLens is a large language model hallucination benchmark introduced by researchers at Meta AI's Fundamental AI Research (FAIR) lab, together with collaborators at the Hong Kong University of Science and…

AI Benchmarks Meta AI

Harness (AI)

Harness, in artificial intelligence, is the software scaffolding that wraps a machine learning model, most often a large language model

AI Agents AI Benchmarks

KernelBench

KernelBench is an AI benchmark and open-source evaluation environment that measures how well large language models can write fast and correct GPU kernels.

AI Benchmarks

LAB-Bench

LAB-Bench (the Language Agent Biology Benchmark) is an AI benchmark of more than 2,400 multiple-choice questions built to measure how well language models and AI agents can perform practical biology research…

AI Benchmarks

LLM Benchmark Comparison (Leaderboard Overview)

As of July 2026, no single model wins every LLM benchmark, but Anthropic's Claude Fable 5 tops the most: it currently leads MMLU-Pro (91.5%), SWE-bench Verified (95.0%), Humanity's Last Exam in both the…

AI Benchmarks Large Language Models

LLM Evaluation

LLM evaluation is the practice of measuring what a large language model can do, how reliably it does it, and how it behaves under adversarial or high-stakes conditions.

AI Benchmarks AI Research

LLM-as-a-judge

LLM-as-a-judge is the practice of using a strong large language model to evaluate the outputs of other models, or of itself, in place of a human annotator.

AI Benchmarks Large Language Models

LM Evaluation Harness

LM Evaluation Harness (the Language Model Evaluation Harness, often abbreviated lm-eval or written lm-evaluation-harness) is an open-source software framework for measuring the performance of large language…

AI Benchmarks Open Source AI

LMArena

LMArena is a crowdsourced artificial intelligence evaluation platform and company that ranks large language models by having anonymous users vote on which of two blind, side-by-side model responses they prefer

AI Benchmarks AI Companies

LongBench v2

LongBench v2 is a benchmark for evaluating how well large language models understand and reason over long contexts.

AI Benchmarks Large Language Models

MATH

MATH is a benchmark of 12,500 competition mathematics problems used to evaluate the mathematical problem-solving ability of machine learning systems, particularly large language models.

AI Benchmarks Reasoning Models

MMLU-Redux

MMLU-Redux is a manually re-annotated, error-corrected subset of the Massive Multitask Language Understanding (MMLU) benchmark

AI Benchmarks

MRCR

MRCR (Multi-Round Co-reference Resolution) is a synthetic long-context evaluation that tests whether a large language model can locate and disambiguate among several near-identical "needles" buried inside a…

AI Benchmarks Large Language Models

MuSR

MuSR (Multistep Soft Reasoning) is a benchmark for evaluating multistep reasoning in large language models, built around long free-text narratives such as murder mysteries, object-placement scenarios, and…

AI Benchmarks Reasoning Models

Multi-SWE-bench

Multi-SWE-bench is a multilingual benchmark for evaluating the ability of large language model based coding systems to resolve real-world software issues across seven programming languages.

AI Benchmarks AI Code Generation

NoLiMa

NoLiMa, short for "No Literal Matching," is a long-context benchmark for large language models that measures how well a model can find and use a single relevant fact buried in a long document when that fact…

AI Benchmarks Large Language Models

OlympiadBench

OlympiadBench is an AI benchmark of 8,476 Olympiad-level mathematics and physics problems, designed to test the advanced scientific reasoning of large language models and large multimodal models.

AI Benchmarks

Omni-MATH

Omni-MATH is an AI benchmark of Olympiad-level competition mathematics, introduced in October 2024 to measure the mathematical reasoning ability of large language models on problems far harder than those in…

AI Benchmarks

Pass@k

Pass@k is the standard metric for evaluating code generation models: it measures the probability that at least one of k generated candidate solutions passes all of a problem's unit tests.

AI Benchmarks AI Code Generation

ProcessBench

ProcessBench is a benchmark for step-level verification of mathematical reasoning, built by the Qwen Team at Alibaba and released in December 2024.

AI Benchmarks Reasoning Models

PutnamBench

PutnamBench is a benchmark for evaluating automated theorem proving systems, especially neural and large-language-model-based provers, on competition mathematics.

AI Benchmarks

RE-Bench

RE-Bench (short for Research Engineering Benchmark) is a benchmark for evaluating the frontier AI research-and-development capabilities of large language model agents, developed by the nonprofit METR (Model…

AI Benchmarks

Real-SWE

Real-SWE is a coding-agent benchmark published in September 2026 by Specific Labs, a San Francisco company that licenses operational data and source code from businesses and packages them as training and…

AI Benchmarks AI Code Generation

RewardBench

RewardBench is a benchmark and public leaderboard for evaluating reward models, the scoring functions that sit at the center of reinforcement learning from human feedback (RLHF).

AI Benchmarks Reinforcement Learning

SWE-bench Verified

SWE-bench Verified is a 500-problem, human-validated subset of the SWE-bench software engineering benchmark, released on August 13

AI Benchmarks AI Code Generation

ScienceAgentBench

ScienceAgentBench is an AI benchmark for evaluating whether language model agents can perform real, data-driven scientific analysis by writing and executing code.

AI Benchmarks

SimpleQA Verified

SimpleQA Verified is a short-form factuality benchmark released by Google DeepMind and Google Research in September 2025 that measures the parametric knowledge of large language models using roughly 1,000…

AI Benchmarks Google DeepMind

Spider 2.0

Spider 2.0 is a benchmark for evaluating large language models on real-world enterprise text-to-SQL workflows.

AI Benchmarks AI Code Generation

StrongREJECT

StrongREJECT is a benchmark and automated evaluator for measuring how well jailbreaking attacks actually work against large language models.

AI Benchmarks AI Safety

SuperGPQA

SuperGPQA is a large graduate-level knowledge and reasoning benchmark for evaluating large language models across 285 academic disciplines.

AI Benchmarks

Task-completion time horizon (METR)

The task-completion time horizon is a metric for AI capability proposed by METR that expresses a model's ability in units of human time: it is the length of task

AI Benchmarks AI Safety

Terminal-Bench

Terminal-Bench is an open benchmark for evaluating AI agents on complex, real-world tasks performed through command-line terminal interfaces.

AI Agents AI Benchmarks

WebArena

WebArena is a realistic, self-hosted web environment and benchmark designed for developing and evaluating autonomous AI agents that perform tasks on the web.

AI Agents AI Benchmarks

WebVoyager

WebVoyager is an end-to-end web agent and its companion benchmark, introduced by Hongliang He and seven coauthors in a paper accepted to ACL 2024 .

AI Agents AI Benchmarks

WildBench

WildBench is an automated evaluation framework for large language models (LLMs) introduced by the Allen Institute for AI (AI2) in 2024.

AI Benchmarks