Simulated annealing
Simulated annealing is a probabilistic method for finding a good approximate solution to a global optimization problem, especially one with many local optima that trap ordinary hill-climbing methods.
Training & Optimization
Explore Training & Optimization through related topics and the articles other pages reference most.
Explore articles
Browse subtopics: Machine Learning
Articles that also belong to these categories. Counts cover all of Training & Optimization.
All articles (145)Machine Learning (70)Deep Learning (41)Large Language Models (18)AI Infrastructure (9)Mathematics (9)AI Inference (8)Reinforcement Learning (8)AI Alignment (7)Developer Tools (7)Natural Language Processing (7)Open Source AI (7)MLOps (6)Statistics (6)NVIDIA (4)AI Hardware (3)Artificial Intelligence (3)Computer Vision (3)Reasoning Models (3)Algorithms (2)Chinese AI (2)Data & Datasets (2)Generative AI (2)Meta AI (2)Model Evaluation (2)Robotics (2)Transformer Models (2)AI Agents (1)AI Code Generation (1)AI Models (1)AI Research (1)AI Tools & Products (1)Enterprise AI (1)Google (1)Google DeepMind (1)Humanoid Robots (1)Microsoft (1)Mixture of Experts (1)Model Architecture (1)Neural Networks (1)OpenAI (1)Small Language Models (1)World Models (1)
Showing 61-70 of 70 articles
Squared Hinge Loss
Squared hinge loss (also called L2 hinge loss or L2-loss) is a loss function used in machine learning for classification tasks, most commonly in support vector machines (SVMs).
Squared Loss
Squared loss, also called quadratic loss, L2 loss, or squared error loss, is a loss function that penalizes a prediction by the square of its error: for a true value $y$ and a prediction $\hat{y}$
Stochastic Gradient Descent (SGD)
Stochastic gradient descent (SGD) is a first-order optimization method that updates parameters using a gradient estimate computed from a randomly selected example or subset of examples.
Supervised fine-tuning
Supervised fine-tuning (SFT) is supervised training applied to a model that has already been trained.
Termination condition
A termination condition, also called a stopping criterion, convergence criterion, or halting condition, is a rule that decides when an iterative algorithm should stop running.
Test-Time Training (TTT)
Test-Time Training (TTT) is a family of machine learning techniques in which a model updates a subset of its own parameters at inference time
Training run
A training run is a single, deliberate instance of training a neural network from scratch (or from a prior checkpoint) on a specified dataset, with a fixed compute budget, hardware allocation, and time horizon.
Wasserstein Loss
Wasserstein loss is a loss function for training generative models that measures the distance between two probability distributions as the Wasserstein-1 distance
Weight Decay
Weight decay is a regularization technique used in training neural networks that shrinks every parameter toward zero by a small fraction on each update step