FormulaOne
FormulaOne is an AI benchmark for evaluating whether a model can design and implement dynamic programming algorithms for graph problems.
Explore Model Evaluation through related topics and the articles other pages reference most.
Articles that also belong to these categories. Counts cover all of Model Evaluation.
Showing 61-120 of 162 articles
FormulaOne is an AI benchmark for evaluating whether a model can design and implement dynamic programming algorithms for graph problems.
FrontierCode is an agentic coding benchmark created by Cognition, the company behind the Devin coding agent.
The Future of Life Institute AI Safety Index (often shortened to the AI Safety Index or FLI AI Safety Index) is a periodic "report card" published by the Future of Life Institute (FLI) that grades the leading…
GDP.pdf is a benchmark from Surge AI that tests whether multimodal language models can answer realistic professional questions about real PDF documents, such as benefits packets, leases, datasheets, clinical…
GDPx is an AI benchmark published by SemiAnalysis on November 2, 2026 to evaluate agents working in simulated professional environments.
Generalization in machine learning is the ability of a trained model to perform accurately on new, unseen data drawn from the same distribution as its training set
A generalization curve (also called a learning curve) is a plot that visualizes how a machine learning model's performance on training data and unseen data changes as a function of some varying quantity, such…
Global-MMLU is a multilingual evaluation benchmark that extends the MMLU question-answering dataset across 42 languages, with designated subsets labeled culturally sensitive (CS) and culturally agnostic (CA).
HELM (Holistic Evaluation of Language Models) is an open-source benchmark framework created by the Center for Research on Foundation Models (CRFM) at Stanford University for the reproducible and transparent…
HELMET (How to Evaluate Long-context Models Effectively and Thoroughly) is a benchmark for evaluating long-context language models introduced by researchers at Princeton University and Intel Labs in 2024.
HalluLens is a large language model hallucination benchmark introduced by researchers at Meta AI's Fundamental AI Research (FAIR) lab, together with collaborators at the Hong Kong University of Science and…
Harness, in artificial intelligence, is the software scaffolding that wraps a machine learning model, most often a large language model
Helicone is an open-source LLM observability platform and AI gateway founded in 2023 by Justin Torre, Cole Gottdank, Barak Oshri, and Scott Nguyen.
Inter-rater agreement is the degree of consensus among two or more independent raters when they label or score the same set of items.
Interpretability in artificial intelligence concerns what people can learn about a system's behavior, predictions, or internal computations, and whether that understanding is reliable enough for a stated…
Intersection over Union (IoU), also known as the Jaccard index or Jaccard similarity coefficient, is the standard overlap metric in object detection and image segmentation: it is the area of overlap between a…
Kullback-Leibler divergence, often abbreviated KL divergence and written…
KernelBench is an AI benchmark and open-source evaluation environment that measures how well large language models can write fast and correct GPU kernels.
LAB-Bench (the Language Agent Biology Benchmark) is an AI benchmark of more than 2,400 multiple-choice questions built to measure how well language models and AI agents can perform practical biology research…
Every figure in this article is a US dollar list rate per 1,000,000 tokens, taken from the provider's own pricing page.
As of July 2026, no single model wins every LLM benchmark, but Anthropic's Claude Fable 5 tops the most: it currently leads MMLU-Pro (91.5%), SWE-bench Verified (95.0%), Humanity's Last Exam in both the…
As of July 2026, the largest context window ever announced belongs to Magic's LTM-2-mini at 100,000,000 tokens (100M), but it is a research prototype that has never been publicly released.
LLM evaluation is the practice of measuring what a large language model can do, how reliably it does it, and how it behaves under adversarial or high-stakes conditions.
LLM-as-a-Verifier is a probabilistic verification framework and open-source Python package for scoring and selecting large language model agent trajectories.
LLM-as-a-judge is the practice of using a strong large language model to evaluate the outputs of other models, or of itself, in place of a human annotator.
LM Evaluation Harness (the Language Model Evaluation Harness, often abbreviated lm-eval or written lm-evaluation-harness) is an open-source software framework for measuring the performance of large language…
LMArena is a crowdsourced artificial intelligence evaluation platform and company that ranks large language models by having anonymous users vote on which of two blind, side-by-side model responses they prefer
LangSmith is a commercial observability, evaluation, and deployment platform for large language model (LLM) applications and AI agents, developed and operated by LangChain Inc. It provides developers and…
Langfuse is an open-source LLM engineering platform that provides observability, tracing, prompt management, evaluation, and dataset tooling for applications built on large language models.
LiveNerf (styled livenerf in its repository) is an open-source, pre-registered benchmark that tracks whether a frontier large language model gets worse after it is released.
LongBench v2 is a benchmark for evaluating how well large language models understand and reason over long contexts.
A loss curve is a plot that shows the value of a loss function over the course of training a machine learning model.
MATH is a benchmark of 12,500 competition mathematics problems used to evaluate the mathematical problem-solving ability of machine learning systems, particularly large language models.
METEOR (Metric for Evaluation of Translation with Explicit ORdering) is an automatic evaluation metric for machine translation and other text-generation tasks that scores a candidate sentence against one or…
MMLU-Redux is a manually re-annotated, error-corrected subset of the Massive Multitask Language Understanding (MMLU) benchmark
MRCR (Multi-Round Co-reference Resolution) is a synthetic long-context evaluation that tests whether a large language model can locate and disambiguate among several near-identical "needles" buried inside a…
Machine-generated text detection is the problem of deciding whether a given passage of text was written by a person or produced by a large language model.
Mean Absolute Error (MAE) is a regression accuracy metric and loss function that measures the average absolute difference between predicted values and actual observed values
Mean Squared Error (MSE), also called mean squared deviation (MSD), is the average of the squared differences between predicted values and actual (observed) values.
Model capacity is the size and richness of the family of functions a machine learning model can represent and learn, which determines how complex a pattern the model can fit.
MuSR (Multistep Soft Reasoning) is a benchmark for evaluating multistep reasoning in large language models, built around long free-text narratives such as murder mysteries, object-placement scenarios, and…
Multi-SWE-bench is a multilingual benchmark for evaluating the ability of large language model based coding systems to resolve real-world software issues across seven programming languages.
MultiLangBench is an AI coding benchmark introduced by OpenAI on October 19, 2026. It evaluates implementations of a SQLite-compatible database engine across 20 programming languages.
NIST ARIA (Assessing Risks and Impacts of AI) is a testing, evaluation, validation, and verification (TEVV) program operated by the United States National Institute of Standards and Technology (NIST) to…
NoLiMa, short for "No Literal Matching," is a long-context benchmark for large language models that measures how well a model can find and use a single relevant fact buried in a long document when that fact…
OlympiadBench is an AI benchmark of 8,476 Olympiad-level mathematics and physics problems, designed to test the advanced scientific reasoning of large language models and large multimodal models.
Omni-MATH is an AI benchmark of Olympiad-level competition mathematics, introduced in October 2024 to measure the mathematical reasoning ability of large language models on problems far harder than those in…
Out-of-bag (OOB) evaluation, sometimes called out-of-bag estimation or OOB error, is a model validation technique used with bagging-based ensemble methods such as random forests and bagged decision trees.
Overfitting is a failure of generalization: a fitted model or a model-selection procedure performs better on the observations used to develop it than on new observations from the population of interest.
PR AUC (Precision-Recall Area Under the Curve), also referred to as AUPRC or AUC-PR, is a classification evaluation metric that quantifies the area beneath a precision-recall curve.
Pass@k is the standard metric for evaluating code generation models: it measures the probability that at least one of k generated candidate solutions passes all of a problem's unit tests.
Patronus AI is an automated LLM evaluation, observability, and guardrails platform founded in 2023 and headquartered in San Francisco.
PhyGenBench (Physics Generation Benchmark) is a benchmark for testing whether text-to-video models produce videos that follow basic physical commonsense.
Physics-IQ Verified is an audited and re-scored version of Physics-IQ, the Google DeepMind benchmark that tests whether video generation models can predict how real physical experiments unfold.
Precision is a classification metric defined as the fraction of positive predictions that are correct: Precision = TP / (TP + FP), where TP is the number of true positives and FP is the number of false…
A precision-recall curve (PR curve) is a graph that plots precision on the y-axis against recall on the x-axis at every possible classification threshold for a binary classification model.
Prediction bias is the difference between the average of a machine learning model's predictions and the average of the ground-truth labels in a dataset.
A process reward model (PRM), also called a process-supervised reward model or step-level verifier, is a learned scoring model that evaluates the correctness or quality of each intermediate step in a large…
ProcessBench is a benchmark for step-level verification of mathematical reasoning, built by the Qwen Team at Alibaba and released in December 2024.
PutnamBench is a benchmark for evaluating automated theorem proving systems, especially neural and large-language-model-based provers, on competition mathematics.