Machine Learning

Deep Learning

Deep learning is a subset of machine learning that uses artificial neural networks with multiple layers to automatically learn representations of data at multiple levels of abstraction

Artificial IntelligenceDeep Learning

Deep Model

A deep model, also called a deep learning model or deep neural network, is an artificial neural network built from many stacked layers of processing units that automatically learns hierarchical representations…

Deep Learning

Deep Neural Network

A deep neural network (DNN) is an artificial neural network with multiple hidden layers of artificial neurons stacked between its input and output layers

Deep LearningNeural Networks

Deep Q-Network (DQN)

Deep Q-Network (DQN) is a reinforcement learning algorithm that uses a deep neural network to approximate the optimal action-value function (Q-function)

Deep LearningReinforcement Learning

DeepLIFT

DeepLIFT (Deep Learning Important FeaTures) is a feature attribution method for deep neural networks introduced by Avanti Shrikumar, Peyton Greenside, and Anshul Kundaje at Stanford University in 2017 .

Deep LearningInterpretability

DeepSpeed

DeepSpeed is an open-source deep learning optimization library, originally developed by Microsoft, that makes distributed training and inference of large models efficient, easy to use, and cost-effective.

AI InfrastructureDeep Learning

Deepak Pathak

Deepak Pathak is an Indian American roboticist and machine learning researcher who is the co-founder and chief executive officer of Skild AI, a startup building a general purpose foundation model for robotics.

PeopleRobotics

Demographic Parity

Demographic parity, also called statistical parity or acceptance rate parity, is a fairness criterion in machine learning that requires a model's predictions to be statistically independent of a protected…

AI Ethics

Denoising

Denoising is the process of removing unwanted noise from data to recover a cleaner underlying signal

Deep Learning

Dense Feature

A dense feature is a feature in machine learning whose vector representation consists mostly or entirely of non-zero values, typically stored as a dense numeric array (a vector or tensor) of floating-point…

Data & Datasets

Dense Layer

A dense layer, also called a fully connected (FC) layer, linear layer, or affine layer, is a layer in an artificial neural network where every input neuron is connected to every output neuron.

Deep LearningNeural Networks

Depth

In machine learning, depth is the number of sequential processing stages a model applies between its input and its output.

Depth up-scaling (DUS)

Depth up-scaling (DUS) is a model-scaling method that builds a deeper large language model by duplicating and stacking the layers of an existing pretrained Transformer checkpoint and then continuing to…

Reinforcement Learning

Depthwise Separable CNN

A depthwise separable convolution is a factorized form of convolution that decomposes a standard convolutional operation into two sequential steps: a depthwise convolution and a pointwise convolution.

Computer VisionModel Architecture

Derived label

A derived label is a label that has been generated programmatically or inferred from other observable signals, rather than collected from direct human annotation of the variable a model is trying to predict.

Data & Datasets

Device

In machine learning, a device is the hardware target on which tensor operations are executed: a CPU, an NVIDIA GPU through CUDA, an Apple GPU through Metal Performance Shaders, a Google TPU through XLA, or an…

Dictionary learning (for interpretability)

Dictionary learning, in the context of mechanistic interpretability, is the framework of decomposing the dense internal activations of a neural network into a sparse, weighted combination drawn from a large

AI Safety

Diederik Kingma

Diederik Kingma is a Dutch machine learning researcher and a founding member of OpenAI who is best known as the first author of the Adam optimizer and the variational autoencoder (VAE)

Deep LearningPeople

Diffusion model

A diffusion model is a generative model that learns to transform samples from a simple reference distribution into samples resembling a data distribution by reversing a gradual corruption process.

Computer VisionDeep Learning

Dimension Reduction

Dimensionality reduction, also known as dimension reduction, is the process of transforming data from a high-dimensional space into a lower-dimensional space while retaining as much of the meaningful structure…

Data & Datasets

Dimensionality reduction

Dimensionality reduction is the process of transforming data from a high-dimensional space into a lower-dimensional representation that retains as much of the meaningful structure of the original data as…

Data Science

Dimensions

In machine learning, the word "dimensions" is overloaded. Depending on the context, it can refer to the number of input features that describe a data point, the number of axes (rank) of a tensor, the width of…

Direct Preference Optimization (DPO)

Direct Preference Optimization (DPO) is a method for aligning large language models with human preferences that replaces the multi-stage reinforcement learning from human feedback (RLHF) pipeline with a single…

AI AlignmentDeep Learning

Discount Factor

The discount factor, almost always written as the Greek letter $$\gamma$$ (gamma), is a scalar hyperparameter in reinforcement learning that controls how much an agent values future rewards relative to…

Reinforcement Learning

Discrete Feature

A discrete feature is a feature (a variable in a dataset) that takes one of a finite or countably infinite set of distinct values, such as a category or an integer count

Data & Datasets

Discriminative Model

A discriminative model is a class of machine learning model that learns the conditional probability distribution P(Y|X) directly, or learns a direct decision boundary mapping inputs X to output labels Y

Discriminator

A discriminator is the neural network in a generative adversarial network (GAN) that is trained to tell real data apart from data produced by the generator

Generative AINeural Networks

Disparate Impact

Disparate impact is a legal and statistical concept describing situations where a seemingly neutral policy, practice, or algorithm produces disproportionately adverse outcomes for members of a protected class…

AI EthicsAI Policy & Regulation

Disparate Treatment

Disparate treatment is the intentional, less favorable treatment of an individual because of a protected attribute such as race, gender, age, religion, national origin, or disability, and in machine learning…

AI EthicsAI Policy & Regulation

Distribution shift

Distribution shift is the condition in which the probability distribution that produced a model's training data differs from the distribution that produces the data the model actually encounters at test or…

AI SafetyData & Datasets

Distributionally Robust Optimization

Distributionally robust optimization (DRO) is a framework for making decisions when the probability distribution of uncertain inputs is not known exactly.

MathematicsStatistics

Divisive Clustering

Divisive clustering, also called top-down clustering, is a hierarchical clustering method that begins with all data points in a single cluster and recursively splits that cluster into smaller subclusters.

DoRA (Weight-Decomposed Low-Rank Adaptation)

DoRA (Weight-Decomposed Low-Rank Adaptation) is a parameter-efficient fine-tuning (PEFT) method for large neural networks introduced in February 2024 by researchers from NVIDIA, the Hong Kong University of…

Training & Optimization

DoReMi

DoReMi (Domain Reweighting with Minimax Optimization) is a method for automatically choosing the proportions, or "domain weights," of each data source in a pretraining corpus for a large language model.

Reinforcement Learning

Domain adaptation

Domain adaptation is the subfield of transfer learning that adapts a model trained on a labelled source domain so it performs well on a related but different target domain, where labels are scarce or absent

Training & Optimization

DoorDash

DoorDash, Inc. is an American on-demand local commerce and food-delivery company that uses applied machine learning to run a real-time logistics marketplace connecting consumers, merchants, and couriers.

AI CompaniesRobotics

Double Descent

Double descent is a phenomenon in machine learning and statistical learning theory in which a model's test error, plotted against increasing model complexity, first traces the classical U-shaped bias-variance…

Deep Learning

Downsampling

Downsampling is the process of reducing the number of samples, the spatial resolution, or the number of data instances in a signal, image, or dataset in order to lower computational cost and memory use while…

Data & DatasetsDeep Learning

Dropout Regularization

Dropout regularization is a regularization technique for neural networks that prevents overfitting by randomly setting a fraction of neuron activations to zero on each training step, forcing the network to…

Deep LearningTraining & Optimization

Dynamic

In machine learning, dynamic describes a model, a training process, or an inference process that runs frequently or continuously on fresh data, as opposed to one that runs once and stays fixed.

Dynamic Programming

Dynamic programming (DP) is an algorithmic technique that solves a complex problem by breaking it into simpler overlapping subproblems, solving each subproblem once, and storing the result so it is never…

AlgorithmsArtificial Intelligence

Dynamic model

A dynamic model in machine learning is a model that is retrained frequently or continuously as new data arrives, so that its parameters track changes in the underlying data distribution over time.

MLOps

Eager Execution

Eager execution is an imperative, define-by-run mode of running machine learning framework operations in which each operation is evaluated immediately as it is called and returns a concrete value

Deep LearningSoftware Development

Early Stopping

Early stopping is a regularization technique that halts the training of an iterative machine learning model as soon as its performance on a held-out validation set stops improving

Deep LearningTraining & Optimization

Earth Mover's Distance

Earth Mover's Distance (EMD), also known as the Wasserstein-1 distance, Kantorovich-Rubinstein metric, or Mallows's distance

Computer VisionMathematics

Edge AI

Edge AI is the practice of running artificial intelligence models directly on the device that generates the data, such as a smartphone, laptop, camera, vehicle, or IoT sensor

AI HardwareArtificial Intelligence

Elastic Net

Elastic Net is a regularization method for linear regression that combines the L1 penalty associated with Lasso regression and the squared L2 penalty associated with ridge regression.

Training & Optimization

Elo rating system (AI model ranking)

The Elo rating system, as applied to AI models, is a method for turning a pile of head-to-head preference votes into a single number per model, so that large language models and chatbots can be ranked on a…

AI BenchmarksModel Evaluation

Embedding Layer

An embedding layer is a neural network component that acts as a trainable lookup table, mapping discrete integer indices (such as word IDs, user IDs, or category codes) to dense, continuous-valued vectors.

Natural Language ProcessingNeural Networks

Embedding Space

An embedding space is a continuous, typically high-dimensional vector space in which data objects (words, sentences, images, users, audio clips, code, or other entities) are represented as dense numerical…

Deep Learning

Embeddings

In machine learning, an embedding is a learned representation that maps an input into a vector, usually so that a downstream model or comparison rule can use relationships encoded in the vector's geometry.

Deep LearningInformation Retrieval

Emergent abilities

Emergent abilities are capabilities of large language models (LLMs) that are absent in smaller models but appear once a model reaches sufficient scale.

AI SafetyLarge Language Models

Emergent misalignment

Emergent misalignment is an AI safety finding, first reported in February 2025, in which fine-tuning a large language model on a single narrow bad behavior causes it to become broadly misaligned across many…

AI Safety

Empirical Risk Minimization

Empirical risk minimization (ERM) is the foundational principle of statistical learning theory: because the true risk (the expected loss over the unknown data distribution) cannot be computed

Training & Optimization

Ensemble

Ensemble methods are techniques in machine learning that combine the predictions of multiple models, known as base learners, to produce a single prediction that is typically more accurate and robust than any…

Ensemble learning

Ensemble learning is a machine learning paradigm that combines multiple models to produce predictions that are better than any individual model could achieve alone.

Entropy

Entropy is a quantitative measure of the uncertainty, randomness, or average information content in a probability distribution: for a discrete random variable it equals the average number of yes/no questions…

Mathematics

Environment

In reinforcement learning (RL), an environment is the external system that an agent interacts with: it receives the agent's actions, transitions to a new state, and returns an observation together with a…

Reinforcement Learning

Episode (Reinforcement Learning)

An episode in reinforcement learning is one complete sequence of interaction between an agent and its environment, starting from an initial state and ending when a terminal state is reached.

Reinforcement Learning

Epoch

An epoch in machine learning is one complete pass through the entire training dataset, during which every example is presented to the model exactly once to compute gradients and update parameters before the…

Deep LearningNeural Networks