AGIEval
AGIEval is an AI benchmark for evaluating foundation models on tasks that were originally designed for, and taken by, humans.
Model Evaluation
Explore Model Evaluation through related topics and the articles other pages reference most.
Most referenced in this topic
Ranked by links from other AI Wiki pages.
Explore articles
Browse subtopics (41)
Articles that also belong to these categories. Counts cover all of Model Evaluation.
All articles (134)AI Benchmarks (56)Machine Learning (55)Statistics (15)AI Companies (14)Large Language Models (12)AI Safety (11)Natural Language Processing (9)AI Agents (6)AI Code Generation (6)Developer Tools (6)MLOps (5)AI Ethics (4)Deep Learning (4)Open Source AI (4)Reasoning Models (4)AI Policy & Regulation (3)Computer Vision (3)Data & Datasets (3)AI Infrastructure (2)AI Research (2)AI Tools & Products (2)Information Retrieval (2)Meta AI (2)Reinforcement Learning (2)Training & Optimization (2)2025 in artificial intelligence (1)AI Models (1)Anthropic (1)Artificial Intelligence (1)Data Centers (1)Education AI (1)Finance AI (1)Google DeepMind (1)Interpretability (1)NVIDIA (1)Neural Networks (1)Research Organizations (1)Software Development (1)Speech & Audio AI (1)Video Generation (1)World Models (1)
Showing 1-60 of 134 articles
ARC Evals
ARC Evals was the evaluations team incubated inside the Alignment Research Center (ARC) between 2022 and 2023, and the direct predecessor of METR (Model Evaluation and Threat Research).
ARC-AGI-2
ARC-AGI-2 (Abstraction and Reasoning Corpus for Artificial General Intelligence 2) is an abstract reasoning benchmark for artificial intelligence, released on March 24, 2025 by the ARC Prize Foundation
AUC (Area Under the ROC Curve)
AUC (Area Under the ROC Curve) is a classifier evaluation metric equal to the probability that a model ranks a randomly chosen positive instance higher than a randomly chosen negative instance.
AUC-ROC
AUC (Area Under the Curve), most often the area under the ROC curve (AUC-ROC), is a threshold-independent evaluation metric that measures how well a binary classification model ranks positive cases above…
Accuracy
Accuracy is a classification metric that measures the fraction of predictions a model gets right: the number of correct predictions divided by the total number of predictions.
Agent evaluation
Agent evaluation is the systematic measurement of how well AI agents (LLM-based systems that plan and act over multiple steps using tools) perform on real-world tasks, using benchmarks, metrics, and testing…
Algorithmic bias
Algorithmic bias is the tendency of a computer system to produce systematic, repeatable errors that advantage some groups of people over others.
Area under the curve
Area under the curve (AUC) is a single scalar metric that summarizes the performance of a binary classifier or diagnostic test across all possible decision thresholds by integrating the area beneath a…
Arena-Hard
Arena-Hard (and its evaluation tool Arena-Hard-Auto) is an automatic large language model (LLM) benchmark developed by the team behind Chatbot Arena that scores instruction-tuned models on 500 challenging
Arize Phoenix
Arize Phoenix is an open-source AI observability and evaluation platform developed by Arize AI for tracing, evaluating, and debugging large language model (LLM) and agent applications.
Average Precision
Average precision (AP) is an evaluation metric that summarizes the precision-recall curve of a ranked list into a single number between 0 and 1, equal to the area under that curve.
BABILong
BABILong is a benchmark for testing how well a large language model can reason over facts scattered through very long text.
BERTScore
BERTScore is an automatic, reference-based metric for evaluating text generation that scores a candidate sentence against one or more references by comparing the contextual embeddings of their tokens rather…
BLEU (Bilingual Evaluation Understudy)
BLEU (Bilingual Evaluation Understudy) is an automatic evaluation metric that scores the quality of machine translation output by measuring how many word sequences (n-grams) it shares with one or more human…
Baseline
In machine learning, a baseline is a simple reference model or method used as a point of comparison to judge whether a more complex model actually adds value.
Benchmark (AI)
In artificial intelligence and machine learning, a benchmark is a specified evaluation used to compare systems under common conditions.
Best AI Models for Reasoning and Math
As of July 2026, the strongest general reasoning models are Anthropic's Claude Opus 4.8 and Claude Fable 5, OpenAI's GPT-5.5, and Google's Gemini 3.1 Pro and Gemini 3 Pro in its Deep Think mode
BigFinanceBench
BigFinanceBench (BFB) is a benchmark for evaluating how well AI agents perform the research work of professional financial analysts, developed by the finance-AI company Rogo and announced on May 27, 2026.
CIDEr
CIDEr (Consensus-based Image Description Evaluation) is an automatic evaluation metric for image captioning that scores a machine-generated caption by how closely it matches the consensus of several human…
CaliBench
CaliBench is a benchmark for image-to-video generative models that asks whether a model reproduces the correct distribution of physical outcomes across many generations from the same starting frame
Calibration Layer
A calibration layer is a post-prediction adjustment appended to a trained machine learning model that rescales its raw output scores or predicted probabilities so they better reflect the true likelihood of…
ChemBench
ChemBench is an automated AI benchmark that measures the chemical knowledge, reasoning, and safety judgment of large language models and compares their performance against expert human chemists.
Classification Threshold
A classification threshold (also called a decision threshold or cut-off point) is a numeric value used to convert the continuous probability output of a classification model into a discrete class label.
ClusterMAX
ClusterMAX is a rating and ranking system for GPU cloud providers published by the research firm SemiAnalysis.
Confusion Matrix
A confusion matrix is a table that summarizes the performance of a classification model by tabulating its predicted class labels against the actual class labels, with correct predictions on the diagonal and…
Coreference Resolution
Coreference resolution is the natural language processing task of finding all the expressions in a text that refer to the same entity and grouping them together.
Cross-Validation
Cross-validation is a statistical resampling technique used in machine learning to estimate how accurately a predictive model will generalize to data it was not trained on.
Cybench
Cybench (short for Cybersecurity benchmark) is an open-source evaluation framework for measuring the cybersecurity capabilities and risks of large language model agents.
Data contamination
Data contamination is the presence of benchmark evaluation data in the corpus used to train a model
Decision Threshold
A decision threshold (also called a classification threshold or cutoff point) is a value used to convert the continuous probability output of a machine learning classifier into a discrete class label.
DesignArena
DesignArena (also written Design Arena) is a crowdsourced benchmark and consumer creation platform for AI-generated design, built by the San Francisco startup Intelligence.
Distribution shift
Distribution shift is the condition in which the probability distribution that produced a model's training data differs from the distribution that produces the data the model actually encounters at test or…
Elo rating system (AI model ranking)
The Elo rating system, as applied to AI models, is a method for turning a pile of head-to-head preference votes into a single number per model, so that large language models and chatbots can be ranked on a…
EnigmaEval
EnigmaEval is an AI benchmark of long, complex multimodal puzzles drawn from real-world puzzle hunts, designed to measure the unstructured, creative, multi-step reasoning abilities of frontier AI models.
Epoch AI
Epoch AI is a nonprofit research organization, founded in 2022 and directed by Jaime Sevilla, that studies the trajectory of artificial intelligence through quantitative analysis of compute, data, algorithms…
Expected calibration error
Expected calibration error (ECE) is a metric that measures how well a classifier's predicted confidence matches its observed accuracy.
F1 score
The F1 score (also written as F1-score, F-score, or F-measure) is the harmonic mean of precision and recall, calculated as $$F_1 = \frac{2 \cdot (\text{Precision} \cdot \text{Recall})}{\text{Precision} +…
FACTS Grounding
FACTS Grounding is a factuality benchmark from Google DeepMind and Google Research that measures whether a large language model answers a request using only the information in a provided source document
FRAMES (benchmark)
FRAMES is an evaluation dataset for retrieval-augmented generation that tests factual accuracy, retrieval, and reasoning together rather than one at a time.
Fairness Metric
A fairness metric is a quantitative, mathematical measure used to evaluate whether a machine learning model's predictions or decisions treat different demographic groups equitably.
False Negative Rate
The false negative rate (FNR), also known as the miss rate, is the proportion of actual positive instances that a model or test incorrectly classifies as negative, computed as FNR = FN / (FN + TP).
False Positive Rate (FPR)
The false positive rate (FPR) is the proportion of actual negative cases that a test, model, or decision process incorrectly classifies as positive, defined as FPR = FP / (FP + TN) where FP is the number of…
False negative
A false negative (FN), also called a Type II error or a miss, is an instance whose true label is positive but that a classification model or test predicts as negative: a real positive case that the model fails…
False positive
A false positive (FP), also called a Type I error or a false alarm, is an instance whose true label is negative but whose predicted label is positive: the classifier raises an alarm where none was warranted.
Feature Importances
Feature importances are numeric scores that quantify how much each input feature contributes to the predictions of a machine learning model.
FormulaOne
FormulaOne is an AI benchmark for evaluating whether a model can design and implement dynamic programming algorithms for graph problems.
Future of Life Institute AI Safety Index
The Future of Life Institute AI Safety Index (often shortened to the AI Safety Index or FLI AI Safety Index) is a periodic "report card" published by the Future of Life Institute (FLI) that grades the leading…
Generalization
Generalization in machine learning is the ability of a trained model to perform accurately on new, unseen data drawn from the same distribution as its training set
Generalization Curve
A generalization curve (also called a learning curve) is a plot that visualizes how a machine learning model's performance on training data and unseen data changes as a function of some varying quantity, such…
Global-MMLU
Global-MMLU is a multilingual evaluation benchmark that extends the MMLU question-answering dataset across 42 languages, with designated subsets labeled culturally sensitive (CS) and culturally agnostic (CA).
HELM (Holistic Evaluation of Language Models)
HELM (Holistic Evaluation of Language Models) is an open-source benchmark framework created by the Center for Research on Foundation Models (CRFM) at Stanford University for the reproducible and transparent…
HELMET
HELMET (How to Evaluate Long-context Models Effectively and Thoroughly) is a benchmark for evaluating long-context language models introduced by researchers at Princeton University and Intel Labs in 2024.
HalluLens
HalluLens is a large language model hallucination benchmark introduced by researchers at Meta AI's Fundamental AI Research (FAIR) lab, together with collaborators at the Hong Kong University of Science and…
Harness (AI)
Harness, in artificial intelligence, is the software scaffolding that wraps a machine learning model, most often a large language model
Helicone
Helicone is an open-source LLM observability platform and AI gateway founded in 2023 by Justin Torre, Cole Gottdank, Barak Oshri, and Scott Nguyen.
Inter-rater agreement
Inter-rater agreement is the degree of consensus among two or more independent raters when they label or score the same set of items.
Interpretability
Interpretability in artificial intelligence concerns what people can learn about a system's behavior, predictions, or internal computations, and whether that understanding is reliable enough for a stated…
IoU
Intersection over Union (IoU), also known as the Jaccard index or Jaccard similarity coefficient, is the standard overlap metric in object detection and image segmentation: it is the area of overlap between a…
KernelBench
KernelBench is an AI benchmark and open-source evaluation environment that measures how well large language models can write fast and correct GPU kernels.