Showing 121-136 of 136 articles
Swarm intelligence
Swarm intelligence (SI) is a branch of artificial intelligence concerned with the collective behavior of decentralized, self-organized systems, in which many...
AI AgentsArtificial Intelligence
Tensor Parallelism
Tensor parallelism (TP) is a distributed training technique that splits the individual weight matrices of a neural network layer across multiple devices, so...
AI InfrastructureLarge Language Models
Termination condition
See also: Machine learning terms A termination condition, also called a stopping criterion, convergence criterion, or halting condition, is a rule that decides...
Machine Learning
Test-Time Training (TTT)
Test-Time Training (TTT) is a family of machine learning techniques in which a model updates a subset of its own parameters at inference time, optimizing a...
AI InferenceMachine Learning
Training loss
In machine learning, training loss is the value of the loss function computed on the training data during model training, and it is the exact quantity that the...
Training run
A training run is a single, deliberate instance of training a neural network from scratch (or from a prior checkpoint) on a specified dataset, with a fixed...
Machine Learning
Unsloth
Unsloth is an open-source Python library that fine-tunes large language models up to two times faster while using up to 70 percent less GPU memory and, in its...
Developer ToolsOpen Source AI
VeRA (Vector-based Random Matrix Adaptation)
Vector-based Random Matrix Adaptation (VeRA) is a parameter-efficient fine-tuning (PEFT) method for adapting large pretrained neural networks to downstream...
Large Language Models
WRAP (Web Rephrase Augmented Pre-training)
WRAP (Web Rephrase Augmented Pre-training) is a synthetic-data pre-training method introduced in the paper "Rephrasing the Web: A Recipe for Compute and...
Data & DatasetsLarge Language Models
Wasserstein Loss
Wasserstein loss is a loss function for training generative models that measures the distance between two probability distributions as the Wasserstein-1...
Generative AIMachine Learning
Weight Decay
Weight decay is a regularization technique used in training neural networks that shrinks every parameter toward zero by a small fraction on each update step,...
Machine Learning
Whole-body control
Whole-body control (WBC) is a class of robotics control techniques that coordinates all of a robot's degrees of freedom at once to achieve multiple prioritized...
Humanoid RobotsRobotics
ZeRO (Zero Redundancy Optimizer)
ZeRO (Zero Redundancy Optimizer) is a family of memory-optimization techniques for training large neural networks introduced by Samyam Rajbhandari, Jeff...
Deep LearningMicrosoft
muP (Maximal Update Parametrization)
muP, short for Maximal Update Parametrization (often written muP, μP, or mu-P), is a parametrization scheme for deep neural networks in which a small set of...
Deep Learning
rsLoRA (Rank-Stabilized LoRA)
rsLoRA (rank-stabilized LoRA) is a one-line modification to the low-rank adaptation fine-tuning method for large neural networks. It replaces the conventional...
Large Language Models
torch.compile
torch.compile is the just-in-time graph capture and compilation feature introduced in PyTorch 2.0, a release first announced at the PyTorch Conference on...
Deep LearningDeveloper Tools