Model merging
Model merging combines the parameters of multiple trained neural networks into a single unified model without any additional training.
Deep Learning
Explore Deep Learning through related topics and the articles other pages reference most.
Explore articles
Browse subtopics: Machine Learning
Articles that also belong to these categories. Counts cover all of Deep Learning.
All articles (333)Machine Learning (187)Neural Networks (81)Computer Vision (53)Training & Optimization (41)Natural Language Processing (37)Generative AI (31)Model Architecture (21)Transformer Models (18)Artificial Intelligence (16)Large Language Models (12)People (12)Reinforcement Learning (12)Developer Tools (11)Mathematics (11)AI Inference (9)Interpretability (8)AI Infrastructure (7)Speech & Audio AI (7)AI Hardware (6)AI Companies (5)AI History (5)Google DeepMind (5)Open Source AI (5)AI Research (4)AI for Science (4)Algorithms (4)Model Evaluation (4)Robotics (4)AI Models (3)Google (3)Meta AI (3)OpenAI (3)AI Alignment (2)AI Tools & Products (2)Chinese AI (2)Data & Datasets (2)Healthcare AI (2)MLOps (2)Microsoft (2)Multimodal AI (2)NVIDIA (2)Software Development (2)Statistics (2)AI Benchmarks (1)AI Ethics (1)AI Safety (1)Data Science (1)Diffusion Models (1)Embodied AI (1)Image Generation (1)Information Retrieval (1)Programming Languages (1)Prompt Engineering (1)Video Generation (1)
Showing 121-180 of 187 articles
Multi-Head Self-Attention
Multi-head self-attention is the core sequence-mixing mechanism of the Transformer architecture: it runs several scaled dot-product attention operations ("heads") in parallel over different learned projections…
Multi-head Latent Attention
Multi-head Latent Attention (MLA) is an attention mechanism for transformer models that achieves a 93.3% reduction in key-value cache size while maintaining or exceeding the performance of traditional…
Multi-task learning
Multi-task learning (MTL) is a machine learning approach in which one model is trained on several tasks at the same time, so that what it learns for one task helps it learn the others.
Multimodal Model
A multimodal model is a machine learning model, or a model-based system, that processes, relates, or produces information across more than one kind of data. Each kind is called a modality.
NaN Trap
A NaN trap (short for "Not a Number" trap) is a failure mode in machine learning training where arithmetic operations produce the special IEEE 754 value NaN, which then propagates through all subsequent…
Neural Machine Translation
Neural machine translation (NMT) is an approach to machine translation in which a single artificial neural network, trained end to end on bilingual text
Neural Network
A neural network (also called an artificial neural network or ANN) is a computational model, loosely inspired by the networks of biological neurons in animal brains
Neural Tangent Kernel
The neural tangent kernel (NTK) is a kernel function built from the parameter gradients of a neural network.
Neural architecture search
Neural architecture search (NAS) is a technique for automating the design of neural network architectures.
OCR Models
OCR Models are artificial intelligence (AI) systems that convert images of typed, handwritten, or printed text into machine-readable digital text through Optical Character Recognition (OCR).
Object detection
Object detection is a computer vision task that finds instances of interest in an image and assigns each one a category.
One-Shot Learning
One-shot learning is a machine learning approach in which a model learns to recognize or classify new categories from only a single labeled example per class.
Optimizer
An optimizer in machine learning is an algorithm that iteratively adjusts a model's learnable parameters to minimize (or maximize) an objective function, commonly called a loss function
Output Layer
The output layer is the final layer of a neural network: it takes the features computed by the hidden layers and converts them into the model's prediction, with its size and activation function fixed by the…
Overfitting
Overfitting is a failure of generalization: a fitted model or a model-selection procedure performs better on the observations used to develop it than on new observations from the population of interest.
Pose estimation
Pose estimation is the computer vision task of detecting and localizing the keypoints (also called landmarks or joints) of a human body, hand, face, animal, or rigid object in images and video, then connecting…
Post-training
Post-training is the stage of large language model (LLM) development that comes after pre-training and turns a raw, general-purpose base model into an aligned, instruction-following AI assistant.
Pre-Trained Model
A pre-trained model is a machine learning model that has already been trained on a large, general-purpose dataset and can then be reused, either as a fixed feature extractor or by fine-tuning
Pre-training
Pre-training is a stage of machine learning in which a model learns parameters from a source dataset or source objective before those parameters are reused or adapted for a target use.
PyTorch
PyTorch is an open-source software library for tensor computation and machine learning.
Quantization
Quantization is the representation of values from a large or continuous set by values from a smaller, usually finite set.
RWKV
RWKV (pronounced "RwaKuv") is an open-source neural network architecture that combines the parallelizable training of Transformers with the constant-time
ReLU
ReLU, short for rectified linear unit, is an activation function that maps a real input to its positive part:
Rectified Linear Unit (ReLU)
The Rectified Linear Unit (ReLU) is the most widely used activation function in deep learning, defined mathematically as $$f(x) = \max(0, x)$$: it returns the input directly when positive and outputs zero…
Recurrent Neural Network
A recurrent neural network (RNN) is a neural network whose computation includes a state that is passed from one step to the next.
Regularization
Regularization is any deliberate change to a learning problem or training algorithm intended to improve performance on data that were not used to fit the model.
Reinforcement Learning from Human Feedback (RLHF)
Reinforcement learning from human feedback (RLHF) is a family of methods in which judgments supplied by people are converted into a reward signal and a policy is optimized with reinforcement learning against…
Reinforcement learning
Reinforcement learning (RL) is a branch of machine learning in which an agent learns to make decisions by taking actions in an environment to maximize a cumulative reward signal, discovering good behavior…
Replay Buffer
A replay buffer (also called an experience replay buffer or replay memory) is a fixed-size memory that stores an off-policy reinforcement learning agent's past transitions, each a tuple of state, action…
Representation
A representation in machine learning is the format in which a data example is encoded before a model processes it: the same image can be represented as raw pixels, a list of hand-designed measurements, or a…
Representation learning
Representation learning is the area of machine learning concerned with getting a system to discover the features it needs for a task, instead of having people specify those features by hand.
Residual connection
A residual connection (also called a skip connection or shortcut connection) is a structural element in neural networks that adds the input of a layer or block directly to its output
RoBERTa
RoBERTa (Robustly Optimized BERT Pretraining Approach) is an open-source natural language processing model released in July 2019 by researchers at Facebook AI (now Meta AI) and the University of Washington…
Robot learning
Robot learning studies how robots acquire or improve behavior from data and experience.
Scaling Laws
Neural scaling laws are empirical relationships that describe how a measured outcome changes as a neural network, its training data, or the computation used to train it becomes larger.
Self-Supervised Learning
Self-supervised learning (SSL) is a machine learning approach in which a model learns representations from unlabeled data by generating its own supervisory signal from the structure of the data itself
Self-attention
Self-attention is a neural network operation in which each position forms a data-dependent mixture of information from positions in the same input sequence or set.
Self-training
Self-training is a semi-supervised learning procedure in which a model trained on a small labeled set is used to generate predictions on unlabeled data
Sequence Model
A sequence model is a class of machine learning models that processes, generates, or predicts ordered data where the position and surrounding context of each element carry meaning.
Sequence-to-Sequence Task
A sequence-to-sequence (seq2seq) task is any machine learning problem in which a model receives a variable-length input sequence and produces a variable-length output sequence, where the input and output may…
Sigmoid Function
The sigmoid function is a mathematical function that maps any real number to a value between 0 and 1 using the formula $$\sigma(x) = \frac{1}{1 + e^{-x}}$$, producing a smooth S-shaped curve.
SimCLR
SimCLR (Simple Framework for Contrastive Learning of Visual Representations) is a self-supervised learning method for computer vision in which a network is trained to recognise that two differently augmented…
Softmax
Softmax is a function that converts a finite vector of real-valued scores into a vector of positive numbers that sum to one.
Sparse attention
Sparse attention is a family of techniques that cut the computational and memory cost of the attention mechanism in transformer models by letting each token attend to only a subset of other tokens in a sequence
Sparse autoencoder
A sparse autoencoder (SAE) is a neural network that adds a sparsity penalty to an autoencoder's training loss so that only a small number of hidden units activate for any given input, producing a wide
Sparsity
Sparsity is the property of a data structure, model, or representation in which most values are zero or near-zero
Speculative Decoding
Speculative decoding is a lossless inference acceleration technique for autoregressive transformer models in which a small, fast draft model proposes several future tokens at once and the larger target model…
Speech Recognition
Speech recognition, usually called automatic speech recognition (ASR), is the computational task of converting a spoken-language signal into a sequence of written symbols.
State space model (deep learning)
A state space model (SSM) in deep learning is a class of sequence model that maps an input sequence to an output sequence through a fixed-size latent state
Stochastic Gradient Descent (SGD)
Stochastic gradient descent (SGD) is a first-order optimization method that updates parameters using a gradient estimate computed from a randomly selected example or subset of examples.
Stride
Stride is the step size by which a filter (or pooling window) moves across the input in a convolutional neural network (CNN): a stride of 1 shifts the filter one position at a time and visits every location…
Supervised fine-tuning
Supervised fine-tuning (SFT) is supervised training applied to a model that has already been trained.
Target Network
A target network is a separate, slowly updated copy of a neural network used in deep reinforcement learning to compute stable learning targets
Tensor
In machine learning, a tensor is a multi-dimensional array of numbers that serves as the fundamental data structure for representing and manipulating data.
Tensor Shape
A tensor shape is a tuple of integers that describes the number of elements along each dimension (or axis) of a tensor.
TensorFlow
TensorFlow is an open-source software platform for numerical computation and machine learning.
Text summarization
Text summarization is the natural language processing (NLP) task of automatically producing a shorter version of one or more documents that preserves the most important information from the original text.
Time Series
A time series is a sequence of observations recorded in time order, often at regular intervals: daily retail sales, hourly electricity demand, per-minute server latency, quarterly GDP.
Token
A token is the basic unit of text that a language model reads and writes: a word, a subword fragment, a single character, or a byte, produced by splitting text during a step called tokenization.