A/B Testing
A/B testing (also called split testing, bucket testing, or an online controlled experiment) is a randomized controlled experiment that compares two variants, a control (A) and a treatment (B), by randomly…
Statistics
Explore Statistics through related topics and the articles other pages reference most.
Explore articles
Browse subtopics: Machine Learning
Articles that also belong to these categories. Counts cover all of Statistics.
All articles (103)Machine Learning (90)Model Evaluation (15)Mathematics (14)Data & Datasets (10)Algorithms (7)Data Science (6)Training & Optimization (6)AI Ethics (5)Computer Science (3)Natural Language Processing (3)AI Benchmarks (2)Artificial Intelligence (2)Deep Learning (2)Reinforcement Learning (2)Robotics (2)AI Alignment (1)AI History (1)AI Inference (1)Computer Vision (1)Diffusion Models (1)Education AI (1)MLOps (1)Neural Networks (1)People (1)Programming Languages (1)
Showing 1-60 of 90 articles
ARIMA
ARIMA (Autoregressive Integrated Moving Average) is a class of statistical models for analyzing and forecasting time series data, specified by three non-negative integer orders written as ARIMA(p, d, q): p is…
AUC-ROC
AUC (Area Under the Curve), most often the area under the ROC curve (AUC-ROC), is a threshold-independent evaluation metric that measures how well a binary classification model ranks positive cases above…
Area under the curve
Area under the curve (AUC) is a single scalar metric that summarizes the performance of a binary classifier or diagnostic test across all possible decision thresholds by integrating the area beneath a…
Bayes' theorem
Bayes' theorem (also called Bayes' rule or Bayes' law) is a fundamental theorem of probability theory that describes how to update the probability of a hypothesis given new evidence.
Bayesian inference
Bayesian inference is a method of statistical inference in which Bayes' theorem is used to update the probability of a hypothesis as new evidence or data becomes available.
Bayesian network
A Bayesian network (also called a belief network, Bayes net, directed graphical model, or probabilistic causal network) is a probabilistic graphical model that represents a set of random variables and their…
Bias-variance tradeoff
The bias-variance tradeoff is a foundational concept in machine learning and statistics that describes the tension between two competing sources of error in predictive models: bias (error from overly…
Calibration (machine learning)
Calibration in machine learning is the property that the probability scores produced by a probabilistic classifier match the empirical frequency of the predicted event: a model that assigns a confidence of 0.8…
Categorical Data
Categorical data, also called qualitative data, is data whose values are discrete labels or groups (such as colors, country names, or blood types) rather than measurable quantities, so they cannot be…
Causal inference
Causal inference is the field of study concerned with drawing conclusions about cause-and-effect relationships from data, answering questions of the form "what would happen to outcome Y if we intervened on…
Concept drift
Concept drift is the change over time in the statistical relationship between a model's inputs and its target, formally when the joint distribution P(X, Y) (and in the most damaging case the conditional P(Y |…
Conditional Random Field
A conditional random field (CRF) is a discriminative probabilistic model for structured prediction, used most often to assign a label to every position in an input sequence.
Continuous Feature
A continuous feature is a numeric input variable in machine learning and statistics that can take any value within a range, including decimals and fractions, rather than a fixed set of categories or counts.
Convenience Sampling
Convenience sampling (also called grab sampling, accidental sampling, or opportunity sampling) is a non-probability sampling method in which data points or participants are selected because they are easy to…
Counterfactual Fairness
Counterfactual fairness is a formal definition of algorithmic fairness rooted in causal inference: a prediction is counterfactually fair toward an individual if it would remain unchanged in a counterfactual…
Curse of Dimensionality
The curse of dimensionality is the set of problems that arise when data has a large number of features (dimensions): as dimensions increase, the volume of the space grows exponentially, the available data…
Data Analysis
Data analysis is the process of inspecting, cleaning, transforming, and modeling data to discover useful information, draw conclusions, and support decision-making.
Data Science
Data science is an interdisciplinary field that uses statistics, programming, and domain expertise to extract knowledge and insights from structured and unstructured data.
Distributionally Robust Optimization
Distributionally robust optimization (DRO) is a framework for making decisions when the probability distribution of uncertain inputs is not known exactly.
Earth Mover's Distance
Earth Mover's Distance (EMD), also known as the Wasserstein-1 distance, Kantorovich-Rubinstein metric, or Mallows's distance
Elo rating system (AI model ranking)
The Elo rating system, as applied to AI models, is a method for turning a pile of head-to-head preference votes into a single number per model, so that large language models and chatbots can be ranked on a…
Estimator
An estimator is a rule, function, or algorithm that takes observed data and produces a value intended to approximate some unknown quantity, typically a parameter of a probability distribution or a function…
Expectation-Maximization (EM) Algorithm
The Expectation-Maximization (EM) algorithm is an iterative method for finding maximum likelihood or maximum a posteriori (MAP) estimates of the parameters of statistical models that involve latent…
Expected calibration error
Expected calibration error (ECE) is a metric that measures how well a classifier's predicted confidence matches its observed accuracy.
Experimenter's Bias
Experimenter's bias (also called the observer-expectancy effect, experimenter expectancy effect, or experimenter effect) is a type of cognitive bias in which a researcher's expectations or beliefs about the…
F1 score
The F1 score (also written as F1-score, F-score, or F-measure) is the harmonic mean of precision and recall, calculated as $$F_1 = \frac{2 \cdot (\text{Precision} \cdot \text{Recall})}{\text{Precision} +…
False Negative Rate
The false negative rate (FNR), also known as the miss rate, is the proportion of actual positive instances that a model or test incorrectly classifies as negative, computed as FNR = FN / (FN + TP).
False Positive Rate (FPR)
The false positive rate (FPR) is the proportion of actual negative cases that a test, model, or decision process incorrectly classifies as positive, defined as FPR = FP / (FP + TN) where FP is the number of…
False negative
A false negative (FN), also called a Type II error or a miss, is an instance whose true label is positive but that a classification model or test predicts as negative: a real positive case that the model fails…
False positive
A false positive (FP), also called a Type I error or a false alarm, is an instance whose true label is negative but whose predicted label is positive: the classifier raises an alarm where none was warranted.
Gaussian Process
A Gaussian process (GP) is a probabilistic machine learning model defined as a collection of random variables, any finite number of which have a joint Gaussian distribution.
Generalized Linear Model
A generalized linear model (GLM) is a flexible extension of ordinary linear regression that allows the response variable to follow any distribution from the exponential family, not just the normal distribution
Independently and Identically Distributed (i.i.d.)
Independently and identically distributed (abbreviated i.i.d., iid, or IID) data is a collection of random variables in which each variable has the same probability distribution as the others and all variables…
Iris dataset
The Iris dataset, sometimes referred to as Fisher's Iris dataset or the Iris flower dataset, is a multivariate dataset introduced by the British statistician and biologist Ronald Fisher in his 1936 paper "The…
KL Divergence (Kullback-Leibler Divergence)
Kullback-Leibler divergence, often abbreviated KL divergence and written $$D_{\mathrm{KL}}(P \parallel Q)$$
Kalman Filter
The Kalman filter is a recursive algorithm that estimates the hidden state of a dynamic system from a sequence of noisy measurements.
L1 Loss
L1 loss is a regression loss function equal to the average of the absolute differences between predicted values and target values, written as $$\frac{1}{n} \sum \lvert y_i - \hat{y}_i \rvert$$.
L2 Loss
L2 loss is the squared-error loss function: for a true value $$y$$ and a predicted value $$\hat{y}$$, it is the squared difference $$(y - \hat{y})^2$$, and averaging it across a dataset gives the mean squared…
Latent Dirichlet allocation
Latent Dirichlet allocation (LDA) is a generative probabilistic model that discovers the hidden thematic structure in a collection of documents by treating each document as a mixture of a small number of…
Least Squares Regression
Least squares regression is a statistical method that fits a model to data by choosing the parameters that minimize the sum of the squared residuals, the squared differences between the observed values and the…
Leo Breiman
Leo Breiman (January 27, 1928 to July 5, 2005) was an American statistician at UC Berkeley whose work on tree-based prediction supplied machine learning with several of its most durable tools.
Linear Discriminant Analysis
Linear Discriminant Analysis (LDA) is a classical statistical method for classification and dimensionality reduction that finds the linear combination of features which best separates two or more classes by…
Linear Regression
Linear regression is a statistical method that models the relationship between one or more independent variables (the predictors or features) and a continuous dependent variable (the response) by fitting a…
Linear model
A linear model is any statistics or machine learning model whose prediction is a linear function of its input features, of the form f(x) = g(w_1 x_1 + w_2 x_2 + ... + w_p x_p + b)
Log-Odds
Log-odds, also known as the logit, is a mathematical transformation that converts a probability value between 0 and 1 into a real number spanning from negative infinity to positive infinity.
Logistic Regression
Logistic regression is a statistical model for a binary response. It represents the conditional probability of one outcome as the logistic transformation of a linear predictor.
Logits
In machine learning and statistics, logits refer to the raw, unnormalized scores output by a model before they are converted into probabilities.
Maximum likelihood estimation (MLE)
Maximum likelihood estimation (MLE) is the method of choosing the parameters of a probability model so that they make the observed data as probable as possible: given a parametric model with density (or mass)…
Mean Absolute Error (MAE)
Mean Absolute Error (MAE) is a regression accuracy metric and loss function that measures the average absolute difference between predicted values and actual observed values
Mean Squared Error (MSE)
Mean Squared Error (MSE), also called mean squared deviation (MSD), is the average of the squared differences between predicted values and actual (observed) values
Multi-Class Logistic Regression
Multi-class logistic regression, also known as multinomial logistic regression, softmax regression, or the maximum entropy (MaxEnt) classifier
Multi-armed bandit
A multi-armed bandit is a sequential decision problem in which a learner repeatedly picks one action from a fixed set, receives a random reward for that action only, and tries to accumulate as much reward as…
Naive Bayes
Naive Bayes is a family of probabilistic classification algorithms that apply Bayes' theorem under a strong ("naive") assumption that every feature is conditionally independent of every other feature given the…
Non-Response Bias
Non-response bias is the error that arises when the people or units that do not respond to a survey, study, or data collection process differ systematically from those that do
Nonstationarity
Nonstationarity refers to the condition in which the statistical properties of a data-generating process change over time.
Outlier Detection
Outlier detection is the process of identifying data points, observations, or patterns that deviate so markedly from the rest of a dataset that they are likely to have been generated by a different process.
Outliers
An outlier is a data point that differs so markedly from the rest of a dataset that it appears not to belong to the same population.
Participation Bias
Participation bias is a systematic error that arises when the individuals who choose to take part in a study, survey, or data collection effort differ in meaningful ways from those who do not, so the resulting…
Particle Filter
A particle filter is a simulation-based method for estimating the changing, unobserved state of a system from a sequence of noisy observations.