Training & Optimization

Explore Training & Optimization through related topics and the articles other pages reference most.

Explore articles

Reset filters
Browse subtopics: Machine Learning

Articles that also belong to these categories. Counts cover all of Training & Optimization.

Showing 61-80 of 80 articles

Pruning

Pruning is a family of techniques used in machine learning and artificial intelligence to remove parts of a model or search space that are estimated to be unnecessary for accuracy or optimality.

AI InferenceMachine Learning

Ridge Regression

Ridge regression is a method of estimating the coefficients of a linear regression model by adding a penalty proportional to the sum of squared coefficients (the squared L2 norm) to the ordinary least squares…

Machine Learning

Ridge Regularization

Ridge regularization, also known as L2 regularization or Tikhonov regularization, is a technique in statistics and machine learning that adds a squared L2-norm penalty to a model's loss function.

Machine LearningStatistics

Shrinkage

Shrinkage in machine learning and statistics is a regularization technique that deliberately pulls model coefficient estimates toward zero, or toward some other fixed shrinkage target

Machine Learning

Simulated annealing

Simulated annealing is a probabilistic method for finding a good approximate solution to a global optimization problem, especially one with many local optima that trap ordinary hill-climbing methods.

AlgorithmsMachine Learning

Squared Hinge Loss

Squared hinge loss (also called L2 hinge loss or L2-loss) is a loss function used in machine learning for classification tasks, most commonly in support vector machines (SVMs).

Machine Learning

Squared Loss

Squared loss, also called quadratic loss, L2 loss, or squared error loss, is a loss function that penalizes a prediction by the square of its error: for a true value $y$ and a prediction $\hat{y}$

Machine LearningStatistics

Termination condition

A termination condition, also called a stopping criterion, convergence criterion, or halting condition, is a rule that decides when an iterative algorithm should stop running.

Machine Learning

Training compute

Training compute is the total number of floating-point operations (FLOP) performed to train a machine learning model, normally counted over the forward and backward passes of the final training run.

AI ComputeMachine Learning

Training run

A training run is a single, deliberate instance of training a neural network from scratch (or from a prior checkpoint) on a specified dataset, with a fixed compute budget, hardware allocation, and time horizon.

Machine Learning

Weight Decay

Weight decay is a regularization technique used in training neural networks that shrinks every parameter toward zero by a small fraction on each update step

Machine Learning