Candidate Sampling
Candidate sampling is a family of training-time optimization techniques used in machine learning to reduce the computational cost of models that must choose among a very large number of output classes.
Training & Optimization
Explore Training & Optimization through related topics and the articles other pages reference most.
Explore articles
Browse subtopics: Natural Language Processing
Articles that also belong to these categories. Counts cover all of Training & Optimization.
All articles (145)Machine Learning (70)Deep Learning (41)Large Language Models (18)AI Infrastructure (9)Mathematics (9)AI Inference (8)Reinforcement Learning (8)AI Alignment (7)Developer Tools (7)Natural Language Processing (7)Open Source AI (7)MLOps (6)Statistics (6)NVIDIA (4)AI Hardware (3)Artificial Intelligence (3)Computer Vision (3)Reasoning Models (3)Algorithms (2)Chinese AI (2)Data & Datasets (2)Generative AI (2)Meta AI (2)Model Evaluation (2)Robotics (2)Transformer Models (2)AI Agents (1)AI Code Generation (1)AI Models (1)AI Research (1)AI Tools & Products (1)Enterprise AI (1)Google (1)Google DeepMind (1)Humanoid Robots (1)Microsoft (1)Mixture of Experts (1)Model Architecture (1)Neural Networks (1)OpenAI (1)Small Language Models (1)World Models (1)
Showing 1-7 of 7 articles
Fill-in-the-middle
Fill-in-the-middle (FIM) is a training objective and inference technique that lets an autoregressive language model generate text for a gap in the middle of a document, conditioned on both the text before the…
LoRA (Low-Rank Adaptation)
Low-Rank Adaptation, usually abbreviated LoRA, is a parameter-efficient fine-tuning method for adapting a pre-trained model.
Next-token prediction
Next-token prediction is the training objective used by most modern language models: the model reads a prefix of tokenized text, outputs a probability distribution over which token comes next, and training…
Pre-Trained Model
A pre-trained model is a machine learning model that has already been trained on a large, general-purpose dataset and can then be reused, either as a fixed feature extractor or by fine-tuning
Pre-training
Pre-training is a stage of machine learning in which a model learns parameters from a source dataset or source objective before those parameters are reused or adapted for a target use.
Supervised fine-tuning
Supervised fine-tuning (SFT) is supervised training applied to a model that has already been trained.