Model Evaluation

Explore Model Evaluation through related topics and the articles other pages reference most.

Explore articles

Browse subtopics (52)

Articles that also belong to these categories. Counts cover all of Model Evaluation.

Showing 61-120 of 162 articles

FormulaOne

FormulaOne is an AI benchmark for evaluating whether a model can design and implement dynamic programming algorithms for graph problems.

AI Benchmarks

Future of Life Institute AI Safety Index

The Future of Life Institute AI Safety Index (often shortened to the AI Safety Index or FLI AI Safety Index) is a periodic "report card" published by the Future of Life Institute (FLI) that grades the leading…

AI Companies

GDP.pdf

GDP.pdf is a benchmark from Surge AI that tests whether multimodal language models can answer realistic professional questions about real PDF documents, such as benefits packets, leases, datasheets, clinical…

AI BenchmarksEnterprise AI

GDPx

GDPx is an AI benchmark published by SemiAnalysis on November 2, 2026 to evaluate agents working in simulated professional environments.

AI AgentsAI Benchmarks

Generalization Curve

A generalization curve (also called a learning curve) is a plot that visualizes how a machine learning model's performance on training data and unseen data changes as a function of some varying quantity, such…

Machine Learning

Global-MMLU

Global-MMLU is a multilingual evaluation benchmark that extends the MMLU question-answering dataset across 42 languages, with designated subsets labeled culturally sensitive (CS) and culturally agnostic (CA).

AI BenchmarksNatural Language Processing

HELMET

HELMET (How to Evaluate Long-context Models Effectively and Thoroughly) is a benchmark for evaluating long-context language models introduced by researchers at Princeton University and Intel Labs in 2024.

AI Benchmarks

HalluLens

HalluLens is a large language model hallucination benchmark introduced by researchers at Meta AI's Fundamental AI Research (FAIR) lab, together with collaborators at the Hong Kong University of Science and…

AI BenchmarksMeta AI

Helicone

Helicone is an open-source LLM observability platform and AI gateway founded in 2023 by Justin Torre, Cole Gottdank, Barak Oshri, and Scott Nguyen.

AI CompaniesDeveloper Tools

Interpretability

Interpretability in artificial intelligence concerns what people can learn about a system's behavior, predictions, or internal computations, and whether that understanding is reliable enough for a stated…

AI EthicsMachine Learning

IoU

Intersection over Union (IoU), also known as the Jaccard index or Jaccard similarity coefficient, is the standard overlap metric in object detection and image segmentation: it is the area of overlap between a…

Computer Vision

KernelBench

KernelBench is an AI benchmark and open-source evaluation environment that measures how well large language models can write fast and correct GPU kernels.

AI Benchmarks

LAB-Bench

LAB-Bench (the Language Agent Biology Benchmark) is an AI benchmark of more than 2,400 multiple-choice questions built to measure how well language models and AI agents can perform practical biology research…

AI Benchmarks

LLM Evaluation

LLM evaluation is the practice of measuring what a large language model can do, how reliably it does it, and how it behaves under adversarial or high-stakes conditions.

AI BenchmarksAI Research

LM Evaluation Harness

LM Evaluation Harness (the Language Model Evaluation Harness, often abbreviated lm-eval or written lm-evaluation-harness) is an open-source software framework for measuring the performance of large language…

AI BenchmarksOpen Source AI

LMArena

LMArena is a crowdsourced artificial intelligence evaluation platform and company that ranks large language models by having anonymous users vote on which of two blind, side-by-side model responses they prefer

AI BenchmarksAI Companies

LangSmith

LangSmith is a commercial observability, evaluation, and deployment platform for large language model (LLM) applications and AI agents, developed and operated by LangChain Inc. It provides developers and…

AI CompaniesAI Infrastructure

Langfuse

Langfuse is an open-source LLM engineering platform that provides observability, tracing, prompt management, evaluation, and dataset tooling for applications built on large language models.

AI CompaniesDeveloper Tools

LiveNerf

LiveNerf (styled livenerf in its repository) is an open-source, pre-registered benchmark that tracks whether a frontier large language model gets worse after it is released.

AI BenchmarksAnthropic

MATH

MATH is a benchmark of 12,500 competition mathematics problems used to evaluate the mathematical problem-solving ability of machine learning systems, particularly large language models.

AI BenchmarksReasoning Models

MMLU-Redux

MMLU-Redux is a manually re-annotated, error-corrected subset of the Massive Multitask Language Understanding (MMLU) benchmark

AI Benchmarks

MRCR

MRCR (Multi-Round Co-reference Resolution) is a synthetic long-context evaluation that tests whether a large language model can locate and disambiguate among several near-identical "needles" buried inside a…

AI BenchmarksLarge Language Models

Model Capacity

Model capacity is the size and richness of the family of functions a machine learning model can represent and learn, which determines how complex a pattern the model can fit.

Machine Learning

MuSR

MuSR (Multistep Soft Reasoning) is a benchmark for evaluating multistep reasoning in large language models, built around long free-text narratives such as murder mysteries, object-placement scenarios, and…

AI BenchmarksReasoning Models

Multi-SWE-bench

Multi-SWE-bench is a multilingual benchmark for evaluating the ability of large language model based coding systems to resolve real-world software issues across seven programming languages.

AI BenchmarksAI Code Generation

MultiLangBench

MultiLangBench is an AI coding benchmark introduced by OpenAI on October 19, 2026. It evaluates implementations of a SQLite-compatible database engine across 20 programming languages.

AI Benchmarks

NIST ARIA

NIST ARIA (Assessing Risks and Impacts of AI) is a testing, evaluation, validation, and verification (TEVV) program operated by the United States National Institute of Standards and Technology (NIST) to…

AI Policy & RegulationAI Safety

NoLiMa

NoLiMa, short for "No Literal Matching," is a long-context benchmark for large language models that measures how well a model can find and use a single relevant fact buried in a long document when that fact…

AI BenchmarksLarge Language Models

OlympiadBench

OlympiadBench is an AI benchmark of 8,476 Olympiad-level mathematics and physics problems, designed to test the advanced scientific reasoning of large language models and large multimodal models.

AI Benchmarks

Omni-MATH

Omni-MATH is an AI benchmark of Olympiad-level competition mathematics, introduced in October 2024 to measure the mathematical reasoning ability of large language models on problems far harder than those in…

AI Benchmarks

Overfitting

Overfitting is a failure of generalization: a fitted model or a model-selection procedure performs better on the observations used to develop it than on new observations from the population of interest.

Deep LearningMachine Learning

PR AUC

PR AUC (Precision-Recall Area Under the Curve), also referred to as AUPRC or AUC-PR, is a classification evaluation metric that quantifies the area beneath a precision-recall curve.

Machine Learning

Pass@k

Pass@k is the standard metric for evaluating code generation models: it measures the probability that at least one of k generated candidate solutions passes all of a problem's unit tests.

AI BenchmarksAI Code Generation

Patronus AI

Patronus AI is an automated LLM evaluation, observability, and guardrails platform founded in 2023 and headquartered in San Francisco.

AI CompaniesAI Safety

Physics-IQ Verified

Physics-IQ Verified is an audited and re-scored version of Physics-IQ, the Google DeepMind benchmark that tests whether video generation models can predict how real physical experiments unfold.

AI BenchmarksVideo Generation

Precision

Precision is a classification metric defined as the fraction of positive predictions that are correct: Precision = TP / (TP + FP), where TP is the number of true positives and FP is the number of false…

Machine Learning

Precision-Recall Curve

A precision-recall curve (PR curve) is a graph that plots precision on the y-axis against recall on the x-axis at every possible classification threshold for a binary classification model.

Machine Learning

Prediction Bias

Prediction bias is the difference between the average of a machine learning model's predictions and the average of the ground-truth labels in a dataset.

Machine Learning

Process reward model (PRM)

A process reward model (PRM), also called a process-supervised reward model or step-level verifier, is a learned scoring model that evaluates the correctness or quality of each intermediate step in a large…

AI SafetyMachine Learning

PutnamBench

PutnamBench is a benchmark for evaluating automated theorem proving systems, especially neural and large-language-model-based provers, on competition mathematics.

AI Benchmarks