LoftQ
LoftQ (short for LoRA-Fine-Tuning-aware Quantization) is a quantization and initialization framework for large language models that jointly quantizes a pre-trained backbone and initializes the attached…
Training & Optimization
Explore Training & Optimization through related topics and the articles other pages reference most.
Explore articles
Browse subtopics (42)
Articles that also belong to these categories. Counts cover all of Training & Optimization.
All articles (145)Machine Learning (70)Deep Learning (41)Large Language Models (18)AI Infrastructure (9)Mathematics (9)AI Inference (8)Reinforcement Learning (8)AI Alignment (7)Developer Tools (7)Natural Language Processing (7)Open Source AI (7)MLOps (6)Statistics (6)NVIDIA (4)AI Hardware (3)Artificial Intelligence (3)Computer Vision (3)Reasoning Models (3)Algorithms (2)Chinese AI (2)Data & Datasets (2)Generative AI (2)Meta AI (2)Model Evaluation (2)Robotics (2)Transformer Models (2)AI Agents (1)AI Code Generation (1)AI Models (1)AI Research (1)AI Tools & Products (1)Enterprise AI (1)Google (1)Google DeepMind (1)Humanoid Robots (1)Microsoft (1)Mixture of Experts (1)Model Architecture (1)Neural Networks (1)OpenAI (1)Small Language Models (1)World Models (1)
Showing 61-120 of 145 articles
Log Loss
Log loss is the negative log-likelihood of the predicted probabilities and the standard loss function for probabilistic classification: for binary labels it is computed as…
LongLoRA
LongLoRA is a parameter-efficient fine-tuning technique that extends the context window of pre-trained large language models with substantially lower computation than full fine-tuning.
Loss
In machine learning, loss (sometimes called error) is a single non-negative number that measures how far a model's prediction is from the correct answer
Loss Curve
A loss curve is a plot that shows the value of a loss function over the course of training a machine learning model.
Loss Function
A loss function assigns a numerical cost to a prediction, decision, or action after an outcome is observed.
Loss Surface
The loss surface (also called the loss landscape, the error surface, or the objective function surface) is the geometric representation of a loss function as a function of the model's parameters.
Masked autoencoder (MAE)
Masked autoencoder (MAE) is a self-supervised learning method for vision transformers that masks roughly 75% of an input image's patches and trains a network to reconstruct the missing pixels from the small…
Megatron-LM
Megatron-LM is NVIDIA's open-source framework for training very large transformer language models across GPU clusters
Mini-batch stochastic gradient descent
Mini-batch stochastic gradient descent (often shortened to mini-batch SGD or MB-SGD) is the optimization algorithm used to train almost every modern neural network: it updates a model's parameters by…
Minimax Loss
Minimax loss is a loss function rooted in game theory and decision theory that measures the worst-case performance of a strategy, algorithm, or model.
Minitron
Minitron is a family of compact language models from NVIDIA, together with the model-compression method used to build them: take one large, already pretrained large language model and shrink it into smaller…
Mistral Forge
Mistral Forge, branded by its developer simply as Forge, is an enterprise platform and engineering service from Mistral AI for building custom large language models and related multimodal models on an…
Model Predictive Control
Model Predictive Control (MPC), also called receding horizon control (RHC), is a feedback control method that, at every sampling instant, solves a finite horizon optimal control problem using the current…
Momentum
Momentum is an optimization technique that accelerates gradient descent by accumulating an exponentially weighted moving average of past gradients into a velocity vector and using that velocity, rather than…
Multi-task learning
Multi-task learning (MTL) is a machine learning approach in which one model is trained on several tasks at the same time, so that what it learns for one task helps it learn the others.
Multi-token prediction
Multi-token prediction (often abbreviated MTP) is a language modeling training objective in which the model is trained to predict several future tokens at each context position rather than only the next token.
Muon (optimizer)
Muon (short for MomentUm Orthogonalized by Newton-Schulz) is a neural-network optimizer that updates the two-dimensional weight matrices of hidden layers by taking the momentum-based SGD update and…
NCCL (NVIDIA Collective Communications Library)
NCCL, the NVIDIA Collective Communications Library, is a library of topology-aware communication primitives for NVIDIA GPU systems.
NaN Trap
A NaN trap (short for "Not a Number" trap) is a failure mode in machine learning training where arithmetic operations produce the special IEEE 754 value NaN, which then propagates through all subsequent…
Next-token prediction
Next-token prediction is the training objective used by most modern language models: the model reads a prefix of tokenized text, outputs a probability distribution over which token comes next, and training…
NormalFloat 4-bit (NF4)
NormalFloat 4-bit (NF4) is a 4-bit numerical data type for storing the weights of deep neural networks, introduced in the 2023 QLoRA paper by Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer…
ORPO
ORPO (Odds Ratio Preference Optimization) is a preference alignment algorithm for large language models that merges supervised fine-tuning and preference alignment into a single training stage, eliminating the…
Objective
In machine learning, an objective (or objective function) is the scalar function that a learning algorithm optimizes during training: the single number whose gradients drive every parameter update.
Objective function
An objective function is the single scalar-valued quantity that an optimization algorithm tries to minimize or maximize during training.
Optimizer
An optimizer in machine learning is an algorithm that iteratively adjusts a model's learnable parameters to minimize (or maximize) an objective function, commonly called a loss function
PEFT
Parameter-efficient fine-tuning (PEFT) is a family of methods that adapt a pre-trained large language model to a downstream task by updating only a small fraction of its parameters
PROWL-1
PROWL-1 is a training framework from the AI lab Odyssey in which a reinforcement learning agent is paid to break a world model.
Parameter Server (PS)
The Parameter Server (PS) is a distributed system architecture for training large machine learning models in which one set of machines, the server nodes, holds the global model parameters as a partitioned…
Parameter update
A parameter update is the step in neural-network training where a model's trainable weights are adjusted using the gradient of the loss
Partitioning strategy
A partitioning strategy in distributed deep learning is the plan that decides how a model and its training data are split across multiple accelerators (typically GPUs or TPUs) so that the workload can be…
PiSSA
PiSSA (Principal Singular values and Singular vectors Adaptation) is a parameter-efficient fine-tuning method for large language models that initializes LoRA-style low-rank adapter matrices from the dominant…
Pipeline Parallelism
Pipeline parallelism (often abbreviated PP) is a distributed training strategy that splits the layers of a deep neural network across multiple accelerator devices so that each device holds one contiguous block…
Pipelining
Pipelining is a term used in two distinct senses within machine learning and artificial intelligence.
Policy gradient methods
Policy gradient methods are a family of reinforcement learning algorithms that directly parameterise the agent's policy and optimise it by stochastic gradient ascent on the expected return.
Pre-Trained Model
A pre-trained model is a machine learning model that has already been trained on a large, general-purpose dataset and can then be reused, either as a fixed feature extractor or by fine-tuning
Pre-training
Pre-training is a stage of machine learning in which a model learns parameters from a source dataset or source objective before those parameters are reused or adapted for a target use.
Proximal Policy Optimization (PPO)
Proximal Policy Optimization (PPO) is an on-policy policy gradient reinforcement learning algorithm that stabilizes training by clipping the policy update so the new policy stays close ("proximal") to the old…
Pruning
Pruning is a family of techniques used in machine learning and artificial intelligence to remove parts of a model or search space that are estimated to be unnecessary for accuracy or optimality.
QLoRA
QLoRA (Quantized Low-Rank Adaptation) is a parameter-efficient fine-tuning method that finetunes a 65-billion-parameter large language model on a single 48 GB GPU while matching the quality of full 16-bit…
RLOO (REINFORCE Leave-One-Out)
RLOO (REINFORCE Leave-One-Out) is an online reinforcement learning algorithm for aligning large language models with reward signals such as those derived from human preferences.
RLVR
Reinforcement Learning with Verifiable Rewards (RLVR) is a post-training paradigm for large language models in which the reward signal comes from a deterministic
RMSProp
RMSProp (Root Mean Square Propagation) is an adaptive learning-rate optimizer that divides each parameter's gradient by a running root-mean-square of that parameter's recent gradients
Regularization
Regularization is any deliberate change to a learning problem or training algorithm intended to improve performance on data that were not used to fit the model.
Regularization Rate
The regularization rate (commonly denoted as $$\lambda$$ or alpha) is a hyperparameter that controls the strength of the penalty applied to a model's parameters during training.
Representation learning
Representation learning is the area of machine learning concerned with getting a system to discover the features it needs for a task, instead of having people specify those features by hand.
Ridge Regression
Ridge regression is a method of estimating the coefficients of a linear regression model by adding a penalty proportional to the sum of squared coefficients (the squared L2 norm) to the ordinary least squares…
Ridge Regularization
Ridge regularization, also known as L2 regularization or Tikhonov regularization, is a technique in statistics and machine learning that adds a squared L2-norm penalty to a model's loss function.
Ring Attention
Ring Attention, formally Ring Attention with Blockwise Transformers, is a distributed algorithm for computing the self-attention operation of transformer neural networks across a ring of compute devices
SOAP (optimizer)
SOAP (ShampoO with Adam in the Preconditioner's eigenbasis) is a second-order optimization algorithm for training deep neural networks, introduced by Nikhil Vyas, Depen Morwani, Rosie Zhao, Mujin Kwun, Itai…
SPIN (Self-Play Fine-Tuning)
SPIN (Self-Play fIne-tuNing) is a post-training method for large language models introduced by researchers at the University of California, Los Angeles (UCLA) in January 2024.
Schedule-Free optimizer
Schedule-Free is a family of optimization algorithms for deep learning and convex stochastic optimization that matches or exceeds the performance of tuned learning-rate schedules without specifying a horizon T…
Self-Instruct
Self-Instruct is a semi-automated framework for aligning a pretrained large language model with instruction-following behavior by bootstrapping its training data from the model itself, introduced in December…
Self-Rewarding Language Models
Self-Rewarding Language Models (SRLM) is an iterative alignment method in which a single large language model alternately plays the role of policy (generating candidate responses to user prompts) and reward…
Self-Taught Evaluator
Self-Taught Evaluator is a method for training a strong LLM-as-a-judge without any human preference annotations, using synthetic training data and an iterative self-improvement loop.
Sequence Parallelism
Sequence parallelism (SP) is a family of distributed training techniques for transformer-based neural networks that partitions activations along the sequence (token) dimension across multiple accelerators…
Shampoo (optimizer)
Shampoo is a second-order stochastic optimization algorithm for training neural networks that maintains, for each parameter tensor
Shrinkage
Shrinkage in machine learning and statistics is a regularization technique that deliberately pulls model coefficient estimates toward zero, or toward some other fixed shrinkage target
SimPO
SimPO (Simple Preference Optimization) is a reference-free offline preference learning algorithm for aligning large language models with human preferences.
Simulated annealing
Simulated annealing is a probabilistic method for finding a good approximate solution to a global optimization problem, especially one with many local optima that trap ordinary hill-climbing methods.