Statistics

A/B Testing

A/B testing (also called split testing, bucket testing, or an online controlled experiment) is a randomized controlled experiment that compares two variants, a...

Data ScienceMachine Learning

ARIMA

ARIMA (Autoregressive Integrated Moving Average) is a class of statistical models for analyzing and forecasting time series data, specified by three...

Machine Learning

AUC-ROC

See also: Machine learning terms AUC (Area Under the Curve), most often the area under the ROC curve (AUC-ROC), is a threshold-independent evaluation metric...

Machine LearningModel Evaluation

Area under the curve

Area under the curve (AUC) is a single scalar metric that summarizes the performance of a binary classifier or diagnostic test across all possible decision...

Machine LearningModel Evaluation

Bayes' theorem

Bayes' theorem (also called Bayes' rule or Bayes' law) is a fundamental theorem of probability theory that describes how to update the probability of a...

Machine LearningMathematics

Bayesian inference

Bayesian inference is a method of statistical inference in which Bayes' theorem is used to update the probability of a hypothesis as new evidence or data...

Machine Learning

Bayesian network

A Bayesian network (also called a belief network, Bayes net, directed graphical model, or probabilistic causal network) is a probabilistic graphical model that...

Machine Learning

Bayesian statistics

Bayesian statistics is a statistical paradigm in which probability expresses a degree of belief that is updated as evidence arrives, using Bayes' theorem. The...

Mathematics

Bias-variance tradeoff

The bias-variance tradeoff is a foundational concept in machine learning and statistics that describes the tension between two competing sources of error in...

Machine Learning

Calibration (machine learning)

Calibration in machine learning is the property that the probability scores produced by a probabilistic classifier match the empirical frequency of the...

Machine Learning

Categorical Data

Categorical data, also called qualitative data, is data whose values are discrete labels or groups (such as colors, country names, or blood types) rather than...

Data & DatasetsMachine Learning

Causal inference

Causal inference is the field of study concerned with drawing conclusions about cause-and-effect relationships from data, answering questions of the form "what...

Machine Learning

Concept drift

Concept drift is the change over time in the statistical relationship between a model's inputs and its target, formally when the joint distribution P(X, Y)...

Data ScienceMLOps

Continuous Feature

A continuous feature is a numeric input variable in machine learning and statistics that can take any value within a range, including decimals and fractions,...

Data & DatasetsMachine Learning

Convenience Sampling

Convenience sampling (also called grab sampling, accidental sampling, or opportunity sampling) is a non-probability sampling method in which data points or...

Data & DatasetsMachine Learning

Counterfactual Fairness

Counterfactual fairness is a formal definition of algorithmic fairness rooted in causal inference: a prediction is counterfactually fair toward an individual...

AI EthicsMachine Learning

Curse of Dimensionality

See also: Machine learning, Feature engineering, Dimensionality reduction The curse of dimensionality is the set of problems that arise when data has a large...

Machine LearningMathematics

Data Analysis

Data analysis is the process of inspecting, cleaning, transforming, and modeling data to discover useful information, draw conclusions, and support...

Data ScienceMachine Learning

Data Science

Data science is an interdisciplinary field that uses statistics, programming, and domain expertise to extract knowledge and insights from structured and...

Computer ScienceEducation AI

Differential privacy

Differential privacy is a mathematical definition of privacy that guarantees the output of an analysis is essentially unchanged whether or not any single...

AI EthicsComputer Science

Earth Mover's Distance

Earth Mover's Distance (EMD), also known as the Wasserstein-1 distance, Kantorovich-Rubinstein metric, or Mallows's distance, is a measure of dissimilarity...

Computer VisionMachine Learning

Elo rating system (AI model ranking)

The Elo rating system, as applied to AI models, is a method for turning a pile of head-to-head preference votes into a single number per model, so that large...

AI BenchmarksMachine Learning

Estimator

An estimator is a rule, function, or algorithm that takes observed data and produces a value intended to approximate some unknown quantity, typically a...

Machine Learning

Expectation-Maximization (EM) Algorithm

The Expectation-Maximization (EM) algorithm is an iterative method for finding maximum likelihood or maximum a posteriori (MAP) estimates of the parameters of...

Machine Learning

Expected calibration error

Expected calibration error (ECE) is a metric that measures how well a classifier's predicted confidence matches its observed accuracy. A model is well...

Machine LearningModel Evaluation

Experimenter's Bias

Experimenter's bias (also called the observer-expectancy effect, experimenter expectancy effect, or experimenter effect) is a type of cognitive bias in which a...

AI EthicsMachine Learning

F1 score

The F1 score (also written as F1-score, F-score, or F-measure) is the harmonic mean of precision and recall, calculated as , and it ranges from 0 (worst) to 1...

Machine LearningModel Evaluation

False Negative Rate

The false negative rate (FNR), also known as the miss rate, is the proportion of actual positive instances that a model or test incorrectly classifies as...

Machine LearningModel Evaluation

False Positive Rate (FPR)

The false positive rate (FPR) is the proportion of actual negative cases that a test, model, or decision process incorrectly classifies as positive, defined as...

Machine LearningModel Evaluation

False negative

A false negative (FN), also called a Type II error or a miss, is an instance whose true label is positive but that a classification model or test predicts as...

Machine LearningModel Evaluation

False positive

A false positive (FP), also called a Type I error or a false alarm, is an instance whose true label is negative but whose predicted label is positive: the...

Machine LearningModel Evaluation

Gaussian Process

A Gaussian process (GP) is a probabilistic machine learning model defined as a collection of random variables, any finite number of which have a joint Gaussian...

Machine Learning

Generalized Linear Model

A generalized linear model (GLM) is a flexible extension of ordinary linear regression that allows the response variable to follow any distribution from the...

Machine Learning

Goodhart's law

Goodhart's law states that "when a measure becomes a target, it ceases to be a good measure": any statistical regularity or metric tends to break down once it...

AI Alignment

Importance sampling

Importance sampling (often abbreviated IS) is a Monte Carlo method for estimating the expectation of a function under a target probability distribution by...

Reinforcement Learning

Independently and Identically Distributed (i.i.d.)

See also: Machine learning terms, Probability, Statistics, Distribution shift, Out-of-distribution detection Independently and identically distributed...

Machine LearningMathematics

Information theory

Information theory is the mathematical study of the quantification, storage, and communication of information, founded by Claude Shannon in his 1948 paper "A...

Mathematics

Inter-rater agreement

See also: Machine learning terms Inter-rater agreement is the degree of consensus among two or more independent raters when they label or score the same set of...

Data & DatasetsModel Evaluation

Iris dataset

The Iris dataset, sometimes referred to as Fisher's Iris dataset or the Iris flower dataset, is a multivariate dataset introduced by the British statistician...

AI BenchmarksData & Datasets

KL Divergence (Kullback-Leibler Divergence)

Kullback-Leibler divergence, often abbreviated KL divergence and written , is a measure of how one probability distribution P differs from a second reference...

Machine LearningMathematics

L1 Loss

L1 loss is a regression loss function equal to the average of the absolute differences between predicted values and target values, written as . It is also...

Machine LearningTraining & Optimization

L2 Loss

L2 loss is the squared-error loss function: for a true value and a predicted value , it is the squared difference , and averaging it across a dataset gives...

Machine LearningTraining & Optimization

Latent Dirichlet allocation

Latent Dirichlet allocation (LDA) is a generative probabilistic model that discovers the hidden thematic structure in a collection of documents by treating...

Artificial IntelligenceMachine Learning

Least Squares Regression

Least squares regression is a statistical method that fits a model to data by choosing the parameters that minimize the sum of the squared residuals, the...

Machine Learning

Linear Discriminant Analysis

Linear Discriminant Analysis (LDA) is a classical statistical method for classification and dimensionality reduction that finds the linear combination of...

Machine Learning

Linear Regression

Linear regression is a statistical method that models the relationship between one or more independent variables (the predictors or features) and a continuous...

Machine Learning

Linear model

A linear model is any statistics or machine learning model whose prediction is a linear function of its input features, of the form f(x) = g(w1 x1 + w2 x2 +...

Machine Learning

Log-Odds

Log-odds, also known as the logit, is a mathematical transformation that converts a probability value between 0 and 1 into a real number spanning from negative...

Machine LearningMathematics

Logistic Regression

See also: Machine learning terms Logistic regression is a statistical method that models the probability of a binary (yes/no) outcome as a function of one or...

Machine Learning

Logits

In machine learning and statistics, logits refer to the raw, unnormalized scores output by a model before they are converted into probabilities. The term has...

Deep LearningMachine Learning

Markov Chain Monte Carlo

Markov Chain Monte Carlo (MCMC) is a class of algorithms for drawing samples from a probability distribution by constructing a Markov chain whose stationary...

Algorithms

Markov chain

A Markov chain is a stochastic process in which the probability of the next state depends only on the current state and not on the sequence of states that came...

Mathematics

Markov property

The Markov property is the condition that a stochastic process is memoryless: the future of the process depends only on its present state, not on the path the...

Maximum likelihood estimation (MLE)

Maximum likelihood estimation (MLE) is the method of choosing the parameters of a probability model so that they make the observed data as probable as...

Machine Learning

Mean Absolute Error (MAE)

Mean Absolute Error (MAE) is a regression accuracy metric and loss function that measures the average absolute difference between predicted values and actual...

Machine LearningModel Evaluation

Mean Squared Error (MSE)

See also: Machine learning terms Mean Squared Error (MSE), also called mean squared deviation (MSD), is the average of the squared differences between...

Machine LearningModel Evaluation

Multi-Class Logistic Regression

See also: Machine learning terms, Logistic regression, Classification Multi-class logistic regression, also known as multinomial logistic regression, softmax...

Machine Learning

Naive Bayes

Naive Bayes is a family of probabilistic classification algorithms that apply Bayes' theorem under a strong ("naive") assumption that every feature is...

Machine Learning

Non-Response Bias

Non-response bias is the error that arises when the people or units that do not respond to a survey, study, or data collection process differ systematically...

Data & DatasetsMachine Learning

Nonstationarity

Nonstationarity refers to the condition in which the statistical properties of a data-generating process change over time. In a stationary process, quantities...

Machine Learning