Machine Learning

Dense Layer

A dense layer, also called a fully connected (FC) layer, linear layer, or affine layer, is a layer in an artificial neural network where every input neuron is...

Deep LearningNeural Networks

Depth

In machine learning, depth is the number of sequential processing stages a model applies between its input and its output. In a neural network it means the...

Depth up-scaling (DUS)

Depth up-scaling (DUS) is a model-scaling method that builds a deeper large language model by duplicating and stacking the layers of an existing pretrained...

Reinforcement Learning

Depthwise Separable CNN

A depthwise separable convolution is a factorized form of convolution that decomposes a standard convolutional operation into two sequential steps: a depthwise...

Computer VisionModel Architecture

Derived label

A derived label is a label that has been generated programmatically or inferred from other observable signals, rather than collected from direct human...

Data & Datasets

Device

In machine learning, a device is the hardware target on which tensor operations are executed: a CPU, an NVIDIA GPU through CUDA, an Apple GPU through Metal...

Dictionary learning (for interpretability)

Dictionary learning, in the context of mechanistic interpretability, is the framework of decomposing the dense internal activations of a neural network into a...

AI Safety

Diederik Kingma

Diederik Kingma is a Dutch machine learning researcher and a founding member of OpenAI who is best known as the first author of the Adam optimizer [1] and the...

Deep LearningPeople

Diffusion model

A diffusion model is a type of generative model that produces data by learning to reverse a gradual noising process: it is trained so that if Gaussian noise is...

Computer VisionDeep Learning

Dimension Reduction

See also: Machine learning terms Dimensionality reduction, also known as dimension reduction, is the process of transforming data from a high-dimensional space...

Data & Datasets

Dimensionality reduction

Dimensionality reduction is the process of transforming data from a high-dimensional space into a lower-dimensional representation that retains as much of the...

Data Science

Dimensions

See also: Machine learning terms In machine learning, the word "dimensions" is overloaded. Depending on the context, it can refer to the number of input...

Direct Preference Optimization (DPO)

Direct Preference Optimization (DPO) is a method for aligning large language models with human preferences that replaces the multi-stage reinforcement learning...

AI AlignmentDeep Learning

Discount Factor

The discount factor, almost always written as the Greek letter (gamma), is a scalar hyperparameter in reinforcement learning that controls how much an agent...

Reinforcement Learning

Discrete Feature

A discrete feature is a feature (a variable in a dataset) that takes one of a finite or countably infinite set of distinct values, such as a category or an...

Data & Datasets

Discriminative Model

See also: Machine learning, Generative model, Classification A discriminative model is a class of machine learning model that learns the conditional...

Discriminator

A discriminator is the neural network in a generative adversarial network (GAN) that is trained to tell real data apart from data produced by the generator,...

Generative AINeural Networks

Disparate Impact

Disparate impact is a legal and statistical concept describing situations where a seemingly neutral policy, practice, or algorithm produces disproportionately...

AI EthicsAI Policy & Regulation

Disparate Treatment

Disparate treatment is the intentional, less favorable treatment of an individual because of a protected attribute such as race, gender, age, religion,...

AI EthicsAI Policy & Regulation

Divisive Clustering

Divisive clustering, also called top-down clustering, is a hierarchical clustering method that begins with all data points in a single cluster and recursively...

DoRA (Weight-Decomposed Low-Rank Adaptation)

DoRA (Weight-Decomposed Low-Rank Adaptation) is a parameter-efficient fine-tuning (PEFT) method for large neural networks introduced in February 2024 by...

Training & Optimization

DoReMi

DoReMi (Domain Reweighting with Minimax Optimization) is a method for automatically choosing the proportions, or "domain weights," of each data source in a...

Reinforcement Learning

Domain adaptation

Domain adaptation is the subfield of transfer learning that adapts a model trained on a labelled source domain so it performs well on a related but different...

Training & Optimization

DoorDash

DoorDash, Inc. is an American on-demand local commerce and food-delivery company that uses applied machine learning to run a real-time logistics marketplace...

AI CompaniesRobotics

Double Descent

Double descent is a phenomenon in machine learning and statistical learning theory in which a model's test error, plotted against increasing model complexity,...

Deep Learning

Downsampling

Downsampling is the process of reducing the number of samples, the spatial resolution, or the number of data instances in a signal, image, or dataset in order...

Data & DatasetsDeep Learning

Dropout Regularization

Dropout regularization is a regularization technique for neural networks that prevents overfitting by randomly setting a fraction of neuron activations to zero...

Deep LearningTraining & Optimization

Dynamic

In machine learning, dynamic describes a model, a training process, or an inference process that runs frequently or continuously on fresh data, as opposed to...

Dynamic Programming

Dynamic programming (DP) is an algorithmic technique that solves a complex problem by breaking it into simpler overlapping subproblems, solving each subproblem...

AlgorithmsArtificial Intelligence

Dynamic model

See also: Machine learning terms A dynamic model in machine learning is a model that is retrained frequently or continuously as new data arrives, so that its...

MLOps

Eager Execution

Eager execution is an imperative, define-by-run mode of running machine learning framework operations in which each operation is evaluated immediately as it is...

Deep LearningSoftware Development

Early Stopping

Early stopping is a regularization technique that halts the training of an iterative machine learning model as soon as its performance on a held-out validation...

Deep LearningTraining & Optimization

Earth Mover's Distance

Earth Mover's Distance (EMD), also known as the Wasserstein-1 distance, Kantorovich-Rubinstein metric, or Mallows's distance, is a measure of dissimilarity...

Computer VisionMathematics

Edge AI

Edge AI is the practice of running artificial intelligence models directly on the device that generates the data, such as a smartphone, laptop, camera,...

AI HardwareArtificial Intelligence

Elastic Net

See also: Regularization, Linear regression Elastic Net is a regularization and variable selection method for linear regression and other generalized linear...

Training & Optimization

Elo rating system (AI model ranking)

The Elo rating system, as applied to AI models, is a method for turning a pile of head-to-head preference votes into a single number per model, so that large...

AI BenchmarksModel Evaluation

Embedding Layer

An embedding layer is a neural network component that acts as a trainable lookup table, mapping discrete integer indices (such as word IDs, user IDs, or...

Natural Language ProcessingNeural Networks

Embedding Space

An embedding space is a continuous, typically high-dimensional vector space in which data objects (words, sentences, images, users, audio clips, code, or other...

Deep Learning

Embeddings

Embeddings are dense vector representations of data in a continuous vector space, where semantically similar items are mapped to nearby points. An embedding...

Deep LearningInformation Retrieval

Emergent abilities

Emergent abilities are capabilities of large language models (LLMs) that are absent in smaller models but appear once a model reaches sufficient scale. Jason...

AI SafetyLarge Language Models

Emergent misalignment

Emergent misalignment is an AI safety finding, first reported in February 2025, in which fine-tuning a large language model on a single narrow bad behavior...

AI Safety

Empirical Risk Minimization

Empirical risk minimization (ERM) is the foundational principle of statistical learning theory: because the true risk (the expected loss over the unknown data...

Training & Optimization

Ensemble

Ensemble methods are techniques in machine learning that combine the predictions of multiple models, known as base learners, to produce a single prediction...

Ensemble learning

Ensemble learning is a machine learning paradigm that combines multiple models to produce predictions that are better than any individual model could achieve...

Entropy

Entropy is a quantitative measure of the uncertainty, randomness, or average information content in a probability distribution: for a discrete random variable...

Mathematics

Environment

See also: Machine learning terms, Environment ChatGPT Plugins In reinforcement learning (RL), an environment is the external system that an agent interacts...

Reinforcement Learning

Episode (Reinforcement Learning)

An episode in reinforcement learning is one complete sequence of interaction between an agent and its environment, starting from an initial state and ending...

Reinforcement Learning

Epoch

An epoch in machine learning is one complete pass through the entire training dataset, during which every example is presented to the model exactly once to...

Deep LearningNeural Networks

Epsilon Greedy Policy

See also: Machine learning terms, Reinforcement Learning, Q-Learning The epsilon-greedy policy (also written as ε-greedy) is a simple action-selection rule for...

Reinforcement Learning

Equality of Opportunity

Equality of opportunity is a group-fairness criterion in machine learning that requires a classifier's true positive rate (TPR) to be equal across all groups...

AI Ethics

Equalized Odds

Equalized odds is a group fairness criterion in machine learning that requires a classifier's true positive rate (TPR) and false positive rate (FPR) to be...

AI Ethics

Estimator

An estimator is a rule, function, or algorithm that takes observed data and produces a value intended to approximate some unknown quantity, typically a...

Statistics

Estimator (tf.estimator)

See also: TensorFlow, Keras, deep learning, machine learning tf.estimator is a high-level TensorFlow API that encapsulates the complete lifecycle of a machine...

Deep LearningDeveloper Tools

Evol-Instruct

Evol-Instruct is a method for automatically generating large instruction tuning datasets by prompting a large language model to rewrite, or "evolve," existing...

Reinforcement Learning

Example

In machine learning, an example is a single data point that a model trains on or makes a prediction about: in supervised settings it is a pair $(x, y)$ where...

Expectation-Maximization (EM) Algorithm

The Expectation-Maximization (EM) algorithm is an iterative method for finding maximum likelihood or maximum a posteriori (MAP) estimates of the parameters of...

Statistics

Expected calibration error

Expected calibration error (ECE) is a metric that measures how well a classifier's predicted confidence matches its observed accuracy. A model is well...

Model EvaluationStatistics

Experience Replay

See also: Reinforcement Learning, Deep Q-Network (DQN), Q-Learning, Replay Buffer Experience replay is a reinforcement learning technique in which an agent...

Reinforcement Learning

Experimenter's Bias

Experimenter's bias (also called the observer-expectancy effect, experimenter expectancy effect, or experimenter effect) is a type of cognitive bias in which a...

AI EthicsStatistics

Exploding Gradient Problem

See also: Machine learning terms, Vanishing gradient problem, Gradient clipping, Backpropagation The exploding gradient problem is a training failure in deep...

Deep LearningNeural Networks