Automatic Differentiation
Automatic differentiation (abbreviated AD, also called algorithmic differentiation, autodiff, or autograd) is a family of techniques for computing exact derivatives of a function specified by a computer program
Machine Learning
Explore learning methods, model architectures, datasets, and practical applications.
Explore articles
Browse subtopics: Mathematics
Articles that also belong to these categories. Counts cover all of Machine Learning.
All articles (858)Deep Learning (187)Natural Language Processing (92)Statistics (90)Training & Optimization (70)Data & Datasets (65)Neural Networks (58)Reinforcement Learning (58)Model Evaluation (55)Artificial Intelligence (54)Large Language Models (49)Mathematics (48)Computer Vision (41)AI Ethics (35)AI Infrastructure (29)AI Safety (26)Open Source AI (25)Data Science (24)AI Benchmarks (22)Developer Tools (20)Algorithms (18)AI Hardware (17)AI Tools & Products (17)Google (16)Generative AI (14)Information Retrieval (14)Model Architecture (13)People (13)AI Agents (12)Interpretability (12)Robotics (12)AI Companies (11)Transformer Models (11)AI Inference (10)MLOps (10)AI Models (8)Computer Science (8)AI Code Generation (7)AI Research (7)Multimodal AI (7)Speech & Audio AI (7)Chinese AI (6)AI Policy & Regulation (5)Meta AI (5)Programming Languages (5)AI History (4)AI for Science (4)Prompt Engineering (4)AI Alignment (3)AI Events (3)Reasoning Models (3)Embodied AI (2)Software Development (2)2025 in artificial intelligence (1)Anthropic (1)Diffusion Models (1)Drug Discovery (1)Education AI (1)Enterprise AI (1)Healthcare AI (1)Image Generation (1)NVIDIA (1)Small Language Models (1)World Models (1)
Showing 1-48 of 48 articles
Bayes' theorem
Bayes' theorem (also called Bayes' rule or Bayes' law) is a fundamental theorem of probability theory that describes how to update the probability of a hypothesis given new evidence.
Bellman Equation
The Bellman equation is a recursive formula stating that the value of a state equals the immediate reward received there plus the discounted value of the next state
Bias (Math) or Bias Term
The bias term is a learnable additive constant b added to the weighted sum of a neuron's inputs before an activation function is applied, computed as $$z = w \cdot x + b$$.
Broadcasting
Broadcasting is the set of rules that lets element-wise operations (addition, subtraction, multiplication, division) act on arrays or tensors of different but compatible shapes by virtually stretching the…
Convergence
Convergence in machine learning is the point at which an iterative optimization algorithm reaches a stable solution, meaning the loss function stops decreasing meaningfully and further parameter updates yield…
Convex Function
A convex function is a real-valued function whose graph curves upward into a bowl or cup shape, so that the line segment (chord) connecting any two points on the graph lies on or above the graph itself.
Convex Optimization
Convex optimization is the branch of mathematical optimization that minimizes a convex function over a convex set, a problem class with one defining advantage: every local minimum is also a global minimum.
Convex Set
A convex set is a set of points in which the line segment connecting any two points of the set lies entirely within the set .
Convolution
Convolution is a mathematical operation that combines two functions to produce a third function, expressing how the shape of one is modified by the other.
Cross-Entropy
Cross-entropy is a measure from information theory of how many bits (or nats) are needed to encode data drawn from a true probability distribution P when using a code optimized for a different
Curse of Dimensionality
The curse of dimensionality is the set of problems that arise when data has a large number of features (dimensions): as dimensions increase, the volume of the space grows exponentially, the available data…
Distributionally Robust Optimization
Distributionally robust optimization (DRO) is a framework for making decisions when the probability distribution of uncertain inputs is not known exactly.
Earth Mover's Distance
Earth Mover's Distance (EMD), also known as the Wasserstein-1 distance, Kantorovich-Rubinstein metric, or Mallows's distance
Entropy
Entropy is a quantitative measure of the uncertainty, randomness, or average information content in a probability distribution: for a discrete random variable it equals the average number of yes/no questions…
Geometric Deep Learning
Geometric deep learning (GDL) is an area of deep learning that designs neural networks around the structure and symmetries of their data.
Gradient
In machine learning, the gradient is the vector of partial derivatives of a loss function with respect to every model parameter, and it points in the direction in which the loss increases most steeply.
Hyperplane
A hyperplane is a flat, affine subspace of dimension n-1 embedded in an n-dimensional space, defined by the linear equation $$w \cdot x + b = 0$$, where w is a normal vector and b is a scalar offset .
Independently and Identically Distributed (i.i.d.)
Independently and identically distributed (abbreviated i.i.d., iid, or IID) data is a collection of random variables in which each variable has the same probability distribution as the others and all variables…
Inductive bias
Inductive bias (also called learning bias) is the set of assumptions that a learning algorithm uses to predict outputs for previously unseen inputs.
Information Gain
Information Gain (IG) is a measure from information theory that quantifies the reduction in entropy (uncertainty) achieved by partitioning a dataset on a particular feature: it equals the entropy of the parent…
KL Divergence (Kullback-Leibler Divergence)
Kullback-Leibler divergence, often abbreviated KL divergence and written $$D_{\mathrm{KL}}(P \parallel Q)$$
Lambda
Lambda (the Greek letter λ) is a symbol used across machine learning, statistics, and computer science to denote several distinct quantities
Linear
In machine learning and mathematics, linear describes a function or relationship in which the output is built from the inputs only through addition and multiplication by constants.
Log Loss
Log loss is the negative log-likelihood of the predicted probabilities and the standard loss function for probabilistic classification: for binary labels it is computed as…
Log-Odds
Log-odds, also known as the logit, is a mathematical transformation that converts a probability value between 0 and 1 into a real number spanning from negative infinity to positive infinity.
Manifold Hypothesis
The manifold hypothesis is the conjecture that real-world high-dimensional data, such as natural images, speech, and text representations, concentrates on or near a manifold of much lower dimension embedded…
Markov Decision Process (MDP)
A Markov Decision Process (MDP) is a mathematical framework for modeling sequential decision-making in stochastic environments, formally defined by the 5-tuple $$(S, A, P, R, \gamma)$$: a set of states, a set…
Matrix factorization
Matrix factorization is a family of mathematical techniques that decompose a matrix into a product of two or more smaller matrices.
Minimax Loss
Minimax loss is a loss function rooted in game theory and decision theory that measures the worst-case performance of a strategy, algorithm, or model.
Nonlinear
Nonlinear describes any function, model, or relationship that does not satisfy the property of linearity.
NumPy
NumPy (short for Numerical Python) is the foundational open-source library for numerical and scientific computing in Python, providing an n-dimensional array object called ndarray along with a large collection…
Partial derivative
A partial derivative measures how a multivariable function changes when one of its inputs is varied while every other input is held fixed.
Perplexity
Perplexity has two distinct meanings in artificial intelligence. In information theory and natural language processing, perplexity (PPL) is an intrinsic score for a probabilistic model: on a specified sequence
Principal Component Analysis (PCA)
Principal component analysis (PCA) is an unsupervised learning technique for dimensionality reduction that identifies the orthogonal directions of maximum variance in high-dimensional data and projects…
Scalar
A scalar is a single numerical value, a quantity with magnitude but no direction, and the simplest object in linear algebra and machine learning.
Shapley value
The Shapley value is a rule from cooperative game theory for dividing the payoff of a group among its members.
Sigmoid Function
The sigmoid function is a mathematical function that maps any real number to a value between 0 and 1 using the formula $$\sigma(x) = \frac{1}{1 + e^{-x}}$$, producing a smooth S-shaped curve.
Similarity Measure
A similarity measure (also called a similarity function or similarity metric) is a real-valued function that quantifies the degree of resemblance between two objects, returning a large value when the objects…
Singular value decomposition
Singular value decomposition (SVD) is a matrix factorization that writes any real or complex m x n matrix $$A$$ as the product $$A = U \Sigma V^\top$$, where U and V are orthogonal matrices (the left and right…
Sparse Vector
A sparse vector is a vector in which most of the elements are zero, in contrast to a dense vector, in which most elements are non-zero.
Statistical learning theory
Statistical learning theory (SLT) is the mathematical framework that explains when and why machine learning algorithms generalize from a finite training sample to unseen data.
Tensor
In machine learning, a tensor is a multi-dimensional array of numbers that serves as the fundamental data structure for representing and manipulating data.
Tensor Rank
The rank of a tensor, also referred to as its order or degree, is the number of dimensions (axes or indices) needed to describe the tensor.
Tensor Shape
A tensor shape is a tuple of integers that describes the number of elements along each dimension (or axis) of a tensor.
VC dimension
The Vapnik-Chervonenkis dimension, almost always shortened to VC dimension, measures the capacity of a class of binary classifiers by asking how many points the class can label in every possible way.
Wasserstein Loss
Wasserstein loss is a loss function for training generative models that measures the distance between two probability distributions as the Wasserstein-1 distance
Weighted Sum
A weighted sum is a mathematical operation that combines multiple input values by multiplying each value by a corresponding weight (coefficient) and then summing the results.