Showing 121-180 of 314 articles
Gradient Accumulation
Gradient accumulation is a deep learning training technique that simulates a large batch size on limited GPU memory by summing the gradients from several small...
Machine LearningTraining & Optimization
Gradient Descent
Gradient descent is a first-order iterative optimization algorithm that minimizes a differentiable loss function by repeatedly stepping in the direction of the...
Machine LearningTraining & Optimization
Gradient checkpointing
Gradient checkpointing, also called activation checkpointing, activation recomputation, or rematerialization, is a memory-saving technique for training deep...
Training & Optimization
Graph Execution
Graph execution is a computation paradigm in machine learning frameworks where mathematical operations are organized into a directed acyclic graph (DAG) before...
Machine LearningSoftware Development
Graph Neural Network
A Graph Neural Network (GNN) is a class of neuralnetwork that operates directly on graph-structured data, meaning data made of nodes (entities) connected by...
Neural Networks
Grokking
Grokking, also called delayed generalization, is a phenomenon in deep learning where a neural network first memorizes its training data (achieving near-perfect...
Machine Learning
Grouped-Query Attention
Grouped-query attention (GQA) is an attention mechanism for transformer language models that partitions the query heads into a small number of groups, where...
Machine LearningTransformer Models
H-Net (dynamic chunking)
H-Net, short for Hierarchical Network, is a tokenizer-free neural sequence model that learns to segment raw bytes into content-adaptive "chunks" as part of...
Neural Networks
Hidden Layer
See also: neural network, input layer, output layer, activation function A hidden layer is a layer of artificial neurons in a neural network that sits between...
Machine LearningNeural Networks
Hyena
Hyena is a sub-quadratic, attention-free neural sequence operator that replaces the self-attention operator of the Transformer with a recurrence of long,...
Model Architecture
Hymba
Hymba is a hybrid-head neural-network architecture for small language models introduced by NVIDIA researchers in November 2024. Its defining idea is to run...
Neural Networks
Hyperparameter
See also: Machine learning terms A hyperparameter is a configuration setting in a machine learning algorithm that is fixed by the practitioner before training...
Machine LearningTraining & Optimization
IP-Adapter
IP-Adapter (short for Image Prompt Adapter) is a lightweight neural network module that adds image-prompt conditioning to a pretrained text-to-image diffusion...
Generative AI
Ian Goodfellow
Ian Goodfellow is an American computer scientist and machine learning researcher best known for inventing the generative adversarial network (GAN) in 2014 and...
Generative AIPeople
Ilya Sutskever
Ilya Sutskever (born December 8, 1986) is a Russian-born, Israeli-Canadian computer scientist who is the co-founder and former Chief Scientist of OpenAI and...
AI SafetyOpenAI
Image Recognition
Image recognition is the field of artificial intelligence and computer vision that enables machines to identify, classify, and interpret the objects, patterns,...
Computer VisionMachine Learning
Image segmentation
Image segmentation is the computer vision task of partitioning a digital image into multiple regions by assigning every pixel a label, producing a pixel-level...
Computer VisionMachine Learning
ImageNet
ImageNet is a large-scale image database of more than 14 million hand-annotated photographs organized into over 21,000 categories according to the WordNet noun...
Computer VisionMachine Learning
In-Context Learning
In-context learning (ICL) is the ability of a large language model to learn a new task at inference time by conditioning on a prompt that contains a few...
Large Language ModelsMachine Learning
Inception (deep learning)
Inception is a family of convolutional neural network (CNN) architectures developed by researchers at Google, first introduced in 2014. The original...
Computer VisionNeural Networks
Inductive bias
Inductive bias (also called learning bias) is the set of assumptions that a learning algorithm uses to predict outputs for previously unseen inputs. Without an...
Machine LearningMathematics
Inference
AI inference is the process of running a fully trained machine learning model on new, previously unseen data to produce a prediction, classification, or...
MLOpsMachine Learning
Input Layer
The input layer is the first layer of a neural network: it receives the raw feature vector for each data sample and passes those values forward to the next...
Machine LearningNeural Networks
Instance segmentation
Instance segmentation is the computer vision task of detecting every object instance in an image and producing a pixel-precise mask for each one, so that two...
Computer Vision
Instruction Tuning
Instruction tuning is the post-pretraining training stage in which a large language model (LLM) is fine-tuned on a curated collection of (instruction,...
Large Language ModelsMachine Learning
Integrated Gradients
Integrated Gradients (IG) is a feature-attribution method for explainable AI that explains a neural network prediction by assigning each input feature an...
InterpretabilityMachine Learning
Iteration
An iteration in machine learning is a single update of a model's parameters during training, performed by processing one batch of data: the model makes...
Machine Learning
JAX
JAX is a high-performance numerical computing library developed by Google that combines a NumPy-compatible interface with composable function transformations...
Machine Learning
Jonathan Ho
Jonathan Ho is a machine learning researcher best known as the lead author of "Denoising Diffusion Probabilistic Models" (DDPM), the 2020 paper that made...
Generative AIPeople
Jürgen Schmidhuber
Jürgen Schmidhuber (anglicized Jurgen Schmidhuber, born 17 January 1963) is a German computer scientist best known as a co-inventor of long short-term memory...
Neural NetworksPeople
KV Cache
A KV cache (key-value cache) is a memory optimization technique used during transformer inference that stores previously computed key and value tensors from...
AI InferenceMachine Learning
Kaiming He
Kaiming He is a Chinese computer scientist known for foundational work in computer vision and deep learning, and is most closely associated with deep residual...
Computer VisionPeople
Keras
Keras is an open-source, high-level neural network API written in Python that lets developers build, train, and deploy deep learning models with minimal code....
AI Tools & ProductsMachine Learning
Knowledge Distillation
Knowledge distillation (also known as model distillation) is a model compression technique in machine learning in which a smaller model, called the student, is...
AI InferenceMachine Learning
Kolmogorov-Arnold Network
A Kolmogorov-Arnold Network (KAN) is a type of neural network architecture proposed as an alternative to the traditional Multi-Layer Perceptron (MLP)....
Machine LearningNeural Networks
Koray Kavukcuoglu
Koray Kavukcuoglu is a Turkish computer scientist who serves as Chief Technology Officer of Google DeepMind and, since June 2025, as Chief AI Architect of...
Google DeepMindPeople
LSTM
Long Short-Term Memory (LSTM) is a recurrent neural network architecture that learns long-range dependencies in sequential data by maintaining a separate,...
Model ArchitectureNeural Networks
Language Model
A language model is a probabilistic model that assigns a probability to a sequence of words or tokens in a natural language, most often by estimating the...
Machine LearningNatural Language Processing
Large Language Model
See also: Machine learning terms, Natural language processing, Transformer A large language model (LLM) is an artificial intelligence system built on a...
Artificial IntelligenceLarge Language Models
Latent diffusion model
A latent diffusion model (LDM) is a type of diffusion model that runs the denoising diffusion process in a compressed latent space learned by a pretrained...
Computer VisionGenerative AI
Latent reasoning via recurrent depth (Huginn)
Latent reasoning via recurrent depth is an approach to scaling a language model's test-time computation by iterating a recurrent transformer block in latent...
Machine Learning
Layer
A layer is the fundamental building block of a neural network: an organized group of neurons (also called nodes or units) that together apply one mathematical...
Machine LearningNeural Networks
Layer normalization
Layer normalization is a technique for normalizing the activations of a neural network across the feature dimension of each individual sample, rather than...
Model Architecture
Layer-wise Relevance Propagation (LRP)
Layer-wise Relevance Propagation (LRP) is an explainable AI method that explains the prediction of a deep neural network by propagating the model's output...
InterpretabilityMachine Learning
LeNet
LeNet is the pioneering family of convolutional neural networks developed by Yann LeCun and collaborators at AT&T Bell Labs between roughly 1988 and 1998 to...
Artificial IntelligenceComputer Vision
Learning Rate
The learning rate is a hyperparameter in machine learning that controls how much a model's parameters change in response to the estimated error each time the...
Machine LearningTraining & Optimization
Lightning Attention
Lightning Attention is an IO-aware (input/output aware) implementation of linear attention that lets the method reach its theoretical linear-time complexity in...
Neural Networks
LoRA (Low-Rank Adaptation)
LoRA (Low-Rank Adaptation) is a parameter-efficient fine-tuning technique that freezes a pre-trained model's weights and trains small, injected low-rank...
Machine LearningNatural Language Processing
Logits
In machine learning and statistics, logits refer to the raw, unnormalized scores output by a model before they are converted into probabilities. The term has...
Machine LearningNeural Networks
Long Short-Term Memory (LSTM)
Long Short-Term Memory (LSTM) is a specialized type of recurrent neural network (RNN) architecture designed to learn long-range dependencies in sequential...
Machine LearningModel Architecture
Loss Curve
See also: Machine learning terms, Loss function A loss curve is a plot that shows the value of a loss function over the course of training a machine learning...
Machine LearningModel Evaluation
Loss Function
See also: Machine learning terms A loss function is a mathematical formula that measures how far a machine learning model's prediction is from the correct...
Machine LearningTraining & Optimization
Loss Surface
The loss surface (also called the loss landscape, the error surface, or the objective function surface) is the geometric representation of a loss function as a...
Machine LearningTraining & Optimization
Machine learning terms/TensorFlow
See also: Machine learning terms TensorFlow is an open-source software library for machine learning, deep learning, and numerical computation, developed and...
Machine Learning
Mamba 2
Mamba 2 is a state space model architecture introduced in the paper "Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured...
AI ModelsModel Architecture
Mamba-3
Mamba-3 is a sequence-modeling architecture in the state space model (SSM) family, introduced in March 2026 by researchers at Carnegie Mellon University and...
Model Architecture
Mask R-CNN
Mask R-CNN is a deep convolutional neural network for instance segmentation, introduced in 2017 by Kaiming He, Georgia Gkioxari, Piotr Dollar, and Ross...
Computer Vision
MaskGIT
MaskGIT, short for Masked Generative Image Transformer, is an image-synthesis method introduced by Google Research in the 2022 paper "MaskGIT: Masked...
Generative AI
Masked Autoregressive (MAR) generation
Masked Autoregressive (MAR) generation is an image-generation method introduced in the 2024 paper "Autoregressive Image Generation without Vector Quantization"...
Generative AI
Masked Language Model
A masked language model (MLM) is a language model trained to predict missing tokens that have been hidden in a sequence of text, using context from both the...
Machine LearningNatural Language Processing