Model Evaluation

Explore Model Evaluation through related topics and the articles other pages reference most.

Explore articles

Browse subtopics (52)

Articles that also belong to these categories. Counts cover all of Model Evaluation.

Showing 121-162 of 162 articles

RE-Bench

RE-Bench (short for Research Engineering Benchmark) is a benchmark for evaluating the frontier AI research-and-development capabilities of large language model agents, developed by the nonprofit METR (Model…

AI Benchmarks

ROUGE

ROUGE (Recall-Oriented Understudy for Gisting Evaluation) is a set of automatic metrics that score the quality of a machine-generated text summary by counting how many overlapping units (n-grams, word…

Machine LearningNatural Language Processing

Ragas

Ragas is an open-source Python framework for evaluating applications built with large language models. It provides evaluation metrics, test-data generation, and tools for comparing experiments.

Developer ToolsInformation Retrieval

Real-SWE

Real-SWE is a coding-agent benchmark published in September 2026 by Specific Labs, a San Francisco company that licenses operational data and source code from businesses and packages them as training and…

AI BenchmarksAI Code Generation

Recall (metric)

Recall is a classification and retrieval metric that measures the proportion of actual positive instances a model correctly identifies, defined as TP / (TP + FN)

Machine Learning

RoboArena

RoboArena is a distributed, real-world benchmark for generalist robot policies in which volunteer evaluators at different laboratories run double-blind, pairwise (A/B) comparisons of policies on tasks and…

AI BenchmarksEmbodied AI

RoboChallenge

RoboChallenge is an online benchmarking platform that evaluates robot control policies, mainly vision-language-action models (VLAs), on physical robots instead of in simulation.

AI BenchmarksEmbodied AI

RoboDojo

RoboDojo is a benchmark for generalist robot manipulation policies that combines a simulation suite of 42 tasks with a real-robot suite of 18 tasks, a standardized physical test platform called…

AI BenchmarksEmbodied AI

Sabotage evaluations

Sabotage evaluations are a set of tests, introduced by Anthropic in October 2024, that try to measure whether a frontier language model is capable of covertly subverting human oversight, decision-making, and…

AI SafetyAnthropic

Sandbagging (artificial intelligence)

Sandbagging, in the context of AI safety, refers to the strategic and intentional underperformance of an AI system on a capability evaluation or specific task, typically to hide a capability from human…

AI Safety

Scale SEAL Leaderboards

The SEAL Leaderboards are a set of expert-curated, contamination-resistant evaluation leaderboards for frontier large language models, produced by the Safety, Evaluations and Alignment Lab (SEAL) at Scale AI.

AI Companies

ScienceAgentBench

ScienceAgentBench is an AI benchmark for evaluating whether language model agents can perform real, data-driven scientific analysis by writing and executing code.

AI Benchmarks

SimpleQA Verified

SimpleQA Verified is a short-form factuality benchmark released by Google DeepMind and Google Research in September 2025 that measures the parametric knowledge of large language models using roughly 1,000…

AI BenchmarksGoogle DeepMind

Splitter

A splitter is a term used in two distinct senses in machine learning. The first and most common sense is a utility that partitions a dataset into subsets such as training, validation, and test sets, or into…

Machine Learning

Stanford HAI AI Index Report

The AI Index Report is an annual, independent, data-driven publication that tracks, distills, and visualizes trends in artificial intelligence across research and development, technical performance, the…

AI Companies

State of AI Report

The State of AI Report is a free, annual review of progress in artificial intelligence, published every year since 2018.

AI Companies

StrongREJECT

StrongREJECT is a benchmark and automated evaluator for measuring how well jailbreaking attacks actually work against large language models.

AI BenchmarksAI Safety

SuperGPQA

SuperGPQA is a large graduate-level knowledge and reasoning benchmark for evaluating large language models across 285 academic disciplines.

AI Benchmarks

Test Set

A test set is a collection of examples reserved from model fitting and model selection so that it can evaluate a fixed machine learning model or a fully specified learning procedure.

Machine Learning

The Leaderboard Illusion

The Leaderboard Illusion is a 2025 research paper, led by Cohere and Cohere Labs with academic collaborators, that argues the most influential public ranking of large language models, Chatbot Arena

AI Companies

True negative

A true negative (TN) is a case that a binary classification model correctly predicts as belonging to the negative class: the true label is negative and the predicted label is also negative.

Machine LearningStatistics

True positive rate (TPR)

The true positive rate (TPR) is the proportion of actual positive cases that a classifier correctly identifies as positive, computed as TPR = TP / (TP + FN), where TP is the number of true positives and FN the…

Machine Learning

Validation Set

A validation set (also called a development set or dev set) is a subset of labeled data that is held out from the training set and used to evaluate a model's performance during development

Machine Learning

Vals AI

Vals AI is an independent, third-party AI model evaluation company based in San Francisco that builds and publishes domain-specific benchmarks measuring how well large language models perform on real…

AI Companies

Vending-Bench

Vending-Bench is a family of benchmarks from Andon Labs that measures the long-term coherence of large language model-based AI agents by tasking them with running a simulated vending-machine business.

AI AgentsAI Benchmarks

WebArena

WebArena is a realistic, self-hosted web environment and benchmark designed for developing and evaluating autonomous AI agents that perform tasks on the web.

AI AgentsAI Benchmarks

WebVoyager

WebVoyager is an end-to-end web agent and its companion benchmark, introduced by Hongliang He and seven coauthors in a paper accepted to ACL 2024.

AI AgentsAI Benchmarks

WildBench

WildBench is an automated evaluation framework for large language models (LLMs) introduced by the Allen Institute for AI (AI2) in 2024.

AI Benchmarks

chrF

chrF is a machine translation evaluation metric that scores a candidate translation by counting the character n-grams it shares with one or more reference translations, then combining character n-gram…

Machine LearningNatural Language Processing