ARC-AGI-2
ARC-AGI-2 (Abstraction and Reasoning Corpus for Artificial General Intelligence 2) is an abstract reasoning benchmark for artificial intelligence, released on March 24, 2025 by the ARC Prize Foundation
Model Evaluation
Explore Model Evaluation through related topics and the articles other pages reference most.
Explore articles
Browse subtopics: Machine Learning
Articles that also belong to these categories. Counts cover all of Model Evaluation.
All articles (134)AI Benchmarks (56)Machine Learning (55)Statistics (15)AI Companies (14)Large Language Models (12)AI Safety (11)Natural Language Processing (9)AI Agents (6)AI Code Generation (6)Developer Tools (6)MLOps (5)AI Ethics (4)Deep Learning (4)Open Source AI (4)Reasoning Models (4)AI Policy & Regulation (3)Computer Vision (3)Data & Datasets (3)AI Infrastructure (2)AI Research (2)AI Tools & Products (2)Information Retrieval (2)Meta AI (2)Reinforcement Learning (2)Training & Optimization (2)2025 in artificial intelligence (1)AI Models (1)Anthropic (1)Artificial Intelligence (1)Data Centers (1)Education AI (1)Finance AI (1)Google DeepMind (1)Interpretability (1)NVIDIA (1)Neural Networks (1)Research Organizations (1)Software Development (1)Speech & Audio AI (1)Video Generation (1)World Models (1)
Showing 1-55 of 55 articles
AUC (Area Under the ROC Curve)
AUC (Area Under the ROC Curve) is a classifier evaluation metric equal to the probability that a model ranks a randomly chosen positive instance higher than a randomly chosen negative instance.
AUC-ROC
AUC (Area Under the Curve), most often the area under the ROC curve (AUC-ROC), is a threshold-independent evaluation metric that measures how well a binary classification model ranks positive cases above…
Accuracy
Accuracy is a classification metric that measures the fraction of predictions a model gets right: the number of correct predictions divided by the total number of predictions.
Algorithmic bias
Algorithmic bias is the tendency of a computer system to produce systematic, repeatable errors that advantage some groups of people over others.
Area under the curve
Area under the curve (AUC) is a single scalar metric that summarizes the performance of a binary classifier or diagnostic test across all possible decision thresholds by integrating the area beneath a…
Average Precision
Average precision (AP) is an evaluation metric that summarizes the precision-recall curve of a ranked list into a single number between 0 and 1, equal to the area under that curve.
BERTScore
BERTScore is an automatic, reference-based metric for evaluating text generation that scores a candidate sentence against one or more references by comparing the contextual embeddings of their tokens rather…
BLEU (Bilingual Evaluation Understudy)
BLEU (Bilingual Evaluation Understudy) is an automatic evaluation metric that scores the quality of machine translation output by measuring how many word sequences (n-grams) it shares with one or more human…
Baseline
In machine learning, a baseline is a simple reference model or method used as a point of comparison to judge whether a more complex model actually adds value.
CIDEr
CIDEr (Consensus-based Image Description Evaluation) is an automatic evaluation metric for image captioning that scores a machine-generated caption by how closely it matches the consensus of several human…
Calibration Layer
A calibration layer is a post-prediction adjustment appended to a trained machine learning model that rescales its raw output scores or predicted probabilities so they better reflect the true likelihood of…
Classification Threshold
A classification threshold (also called a decision threshold or cut-off point) is a numeric value used to convert the continuous probability output of a classification model into a discrete class label.
Confusion Matrix
A confusion matrix is a table that summarizes the performance of a classification model by tabulating its predicted class labels against the actual class labels, with correct predictions on the diagonal and…
Coreference Resolution
Coreference resolution is the natural language processing task of finding all the expressions in a text that refer to the same entity and grouping them together.
Cross-Validation
Cross-validation is a statistical resampling technique used in machine learning to estimate how accurately a predictive model will generalize to data it was not trained on.
Decision Threshold
A decision threshold (also called a classification threshold or cutoff point) is a value used to convert the continuous probability output of a machine learning classifier into a discrete class label.
Distribution shift
Distribution shift is the condition in which the probability distribution that produced a model's training data differs from the distribution that produces the data the model actually encounters at test or…
Elo rating system (AI model ranking)
The Elo rating system, as applied to AI models, is a method for turning a pile of head-to-head preference votes into a single number per model, so that large language models and chatbots can be ranked on a…
Expected calibration error
Expected calibration error (ECE) is a metric that measures how well a classifier's predicted confidence matches its observed accuracy.
F1 score
The F1 score (also written as F1-score, F-score, or F-measure) is the harmonic mean of precision and recall, calculated as $$F_1 = \frac{2 \cdot (\text{Precision} \cdot \text{Recall})}{\text{Precision} +…
Fairness Metric
A fairness metric is a quantitative, mathematical measure used to evaluate whether a machine learning model's predictions or decisions treat different demographic groups equitably.
False Negative Rate
The false negative rate (FNR), also known as the miss rate, is the proportion of actual positive instances that a model or test incorrectly classifies as negative, computed as FNR = FN / (FN + TP).
False Positive Rate (FPR)
The false positive rate (FPR) is the proportion of actual negative cases that a test, model, or decision process incorrectly classifies as positive, defined as FPR = FP / (FP + TN) where FP is the number of…
False negative
A false negative (FN), also called a Type II error or a miss, is an instance whose true label is positive but that a classification model or test predicts as negative: a real positive case that the model fails…
False positive
A false positive (FP), also called a Type I error or a false alarm, is an instance whose true label is negative but whose predicted label is positive: the classifier raises an alarm where none was warranted.
Feature Importances
Feature importances are numeric scores that quantify how much each input feature contributes to the predictions of a machine learning model.
Generalization
Generalization in machine learning is the ability of a trained model to perform accurately on new, unseen data drawn from the same distribution as its training set
Generalization Curve
A generalization curve (also called a learning curve) is a plot that visualizes how a machine learning model's performance on training data and unseen data changes as a function of some varying quantity, such…
Interpretability
Interpretability in artificial intelligence concerns what people can learn about a system's behavior, predictions, or internal computations, and whether that understanding is reliable enough for a stated…
Loss Curve
A loss curve is a plot that shows the value of a loss function over the course of training a machine learning model.
METEOR (metric)
METEOR (Metric for Evaluation of Translation with Explicit ORdering) is an automatic evaluation metric for machine translation and other text-generation tasks that scores a candidate sentence against one or…
Mean Absolute Error (MAE)
Mean Absolute Error (MAE) is a regression accuracy metric and loss function that measures the average absolute difference between predicted values and actual observed values
Mean Squared Error (MSE)
Mean Squared Error (MSE), also called mean squared deviation (MSD), is the average of the squared differences between predicted values and actual (observed) values
Model Capacity
Model capacity is the size and richness of the family of functions a machine learning model can represent and learn, which determines how complex a pattern the model can fit.
Out-of-bag evaluation (OOB evaluation)
Out-of-bag (OOB) evaluation, sometimes called out-of-bag estimation or OOB error, is a model validation technique used with bagging-based ensemble methods such as random forests and bagged decision trees.
Overfitting
Overfitting is a failure of generalization: a fitted model or a model-selection procedure performs better on the observations used to develop it than on new observations from the population of interest.
PR AUC
PR AUC (Precision-Recall Area Under the Curve), also referred to as AUPRC or AUC-PR, is a classification evaluation metric that quantifies the area beneath a precision-recall curve.
Pass@k
Pass@k is the standard metric for evaluating code generation models: it measures the probability that at least one of k generated candidate solutions passes all of a problem's unit tests.
Precision
Precision is a classification metric defined as the fraction of positive predictions that are correct: Precision = TP / (TP + FP), where TP is the number of true positives and FP is the number of false…
Precision-Recall Curve
A precision-recall curve (PR curve) is a graph that plots precision on the y-axis against recall on the x-axis at every possible classification threshold for a binary classification model.
Prediction Bias
Prediction bias is the difference between the average of a machine learning model's predictions and the average of the ground-truth labels in a dataset.
Process reward model (PRM)
A process reward model (PRM), also called a process-supervised reward model or step-level verifier, is a learned scoring model that evaluates the correctness or quality of each intermediate step in a large…
ROC (Receiver Operating Characteristic) Curve
A Receiver Operating Characteristic (ROC) curve is a graph that measures how well a binary classification system separates two classes by plotting its true positive rate (TPR, also called sensitivity or…
ROUGE
ROUGE (Recall-Oriented Understudy for Gisting Evaluation) is a set of automatic metrics that score the quality of a machine-generated text summary by counting how many overlapping units (n-grams, word…
Recall (metric)
Recall is a classification and retrieval metric that measures the proportion of actual positive instances a model correctly identifies, defined as TP / (TP + FN)
Root Mean Squared Error (RMSE)
Root Mean Squared Error (RMSE), also known as root mean square deviation (RMSD), is a regression evaluation metric equal to the square root of the average of the squared differences between predicted and…
Splitter
A splitter is a term used in two distinct senses in machine learning. The first and most common sense is a utility that partitions a dataset into subsets such as training, validation, and test sets, or into…
Test Set
A test set is a collection of examples reserved from model fitting and model selection so that it can evaluate a fixed machine learning model or a fully specified learning procedure.
True negative
A true negative (TN) is a case that a binary classification model correctly predicts as belonging to the negative class: the true label is negative and the predicted label is also negative.
True positive
A true positive (TP) is a prediction that is correctly positive: the model predicts the positive class and the true label is also positive.
True positive rate (TPR)
The true positive rate (TPR) is the proportion of actual positive cases that a classifier correctly identifies as positive, computed as TPR = TP / (TP + FN), where TP is the number of true positives and FN the…
Validation Set
A validation set (also called a development set or dev set) is a subset of labeled data that is held out from the training set and used to evaluate a model's performance during development
Word error rate
Word error rate (WER) is the standard metric for measuring the accuracy of an automatic speech recognition (ASR) system
chrF
chrF is a machine translation evaluation metric that scores a candidate translation by counting the character n-grams it shares with one or more reference translations, then combining character n-gram…