Candidate Sampling
Candidate sampling is a family of training-time optimization techniques used in machine learning to reduce the computational cost of models that must choose among a very large number of output classes.
Natural Language Processing
Explore Natural Language Processing through related topics and the articles other pages reference most.
Explore articles
Browse subtopics: Training & Optimization
Articles that also belong to these categories. Counts cover all of Natural Language Processing.
All articles (239)Machine Learning (92)Large Language Models (58)AI Benchmarks (42)Deep Learning (37)Information Retrieval (29)AI Models (24)Artificial Intelligence (24)Open Source AI (23)Data & Datasets (15)AI Companies (9)Model Evaluation (9)Transformer Models (9)Meta AI (8)Speech & Audio AI (8)OpenAI (7)Training & Optimization (7)AI Ethics (6)AI Safety (6)AI Tools & Products (6)Computer Vision (6)Generative AI (6)People (6)AI Agents (5)Prompt Engineering (5)Developer Tools (4)Neural Networks (4)AI Code Generation (3)Algorithms (3)Anthropic (3)Mathematics (3)Reasoning Models (3)Statistics (3)AI Alignment (2)AI History (2)AI Research (2)Chinese AI (2)Diffusion Models (2)Google DeepMind (2)Healthcare AI (2)Model Architecture (2)Multimodal AI (2)AI Inference (1)Computer Science (1)Education AI (1)Enterprise AI (1)Legal AI (1)Microsoft (1)Mixture of Experts (1)NVIDIA (1)Research Organizations (1)Voice AI (1)
Showing 1-7 of 7 articles
Fill-in-the-middle
Fill-in-the-middle (FIM) is a training objective and inference technique that lets an autoregressive language model generate text for a gap in the middle of a document, conditioned on both the text before the…
LoRA (Low-Rank Adaptation)
Low-Rank Adaptation, usually abbreviated LoRA, is a parameter-efficient fine-tuning method for adapting a pre-trained model.
Next-token prediction
Next-token prediction is the training objective used by most modern language models: the model reads a prefix of tokenized text, outputs a probability distribution over which token comes next, and training…
Pre-Trained Model
A pre-trained model is a machine learning model that has already been trained on a large, general-purpose dataset and can then be reused, either as a fixed feature extractor or by fine-tuning
Pre-training
Pre-training is a stage of machine learning in which a model learns parameters from a source dataset or source objective before those parameters are reused or adapted for a target use.
Supervised fine-tuning
Supervised fine-tuning (SFT) is supervised training applied to a model that has already been trained.