Training & Optimization
137 articles
AdaGrad
Adafactor
Deep Learning
Adam optimizer
AdamW
Machine Learning
AutoML (Automated Machine Learning)
Developer Tools, MLOps, Model Architecture
Axolotl
Developer Tools, Open Source AI
Bayesian Optimization
Machine Learning
Candidate Sampling
Machine Learning, Natural Language Processing, Neural Networks
Clipping
Deep Learning, Machine Learning
Context Parallelism
AI Infrastructure
Convergence
Machine Learning, Mathematics
Convex Function
Machine Learning, Mathematics
Convex Optimization
Machine Learning, Mathematics
Convex Set
Machine Learning, Mathematics
Cosine learning rate schedule
Deep Learning
Cost
Curriculum learning
Deep Learning, Machine Learning
DPO
AI Alignment
DeepSeek-R1-Distill
AI Models, Chinese AI, Large Language Models
DeepSpeed
AI Infrastructure, Deep Learning, Machine Learning
DiLoCo
Google DeepMind
Distributed training
MLOps
DoRA (Weight-Decomposed Low-Rank Adaptation)
Machine Learning
Domain adaptation
Machine Learning
Dropout
Deep Learning
Dropout Regularization
Deep Learning, Machine Learning
Early Stopping
Deep Learning, Machine Learning
Elastic Net
Machine Learning
Empirical Risk Minimization
Machine Learning
Expert Parallelism
AI Infrastructure, Mixture of Experts
FP4 (4-bit floating point)
AI Hardware, AI Inference
Fine Tuning
Deep Learning, Machine Learning
Focal loss
Computer Vision, Deep Learning, Machine Learning
Fully Sharded Data Parallel (FSDP)
AI Infrastructure, Deep Learning, Developer Tools
GRPO
AI Inference, Chinese AI, Reasoning Models
GaLore (Gradient Low-Rank Projection)
Machine Learning
Gradient
Machine Learning, Mathematics
Gradient Accumulation
Deep Learning, Machine Learning
Gradient Descent
Deep Learning, Machine Learning
Gradient checkpointing
Deep Learning
Gradient clipping
Hinge Loss
Machine Learning
HuggingFace PEFT
Developer Tools, Open Source AI
HuggingFace TRL
Open Source AI, Reinforcement Learning
Hyperparameter
Deep Learning, Machine Learning
InstructGPT
AI Alignment, Large Language Models, OpenAI
KTO
AI Alignment, AI Inference, Reinforcement Learning
L0 Regularization
Machine Learning
L1 Loss
Machine Learning, Statistics
L1 Regularization
Machine Learning
L2 Loss
Machine Learning, Statistics
L2 Regularization
Machine Learning
LIMA (Less Is More for Alignment)
AI Research, Meta AI
LLaMA-Factory
Developer Tools, Open Source AI
Lasso Regression
Machine Learning
Learning Rate
Deep Learning, Machine Learning
Lion (optimizer)
Algorithms, Google
LoRA (Low-Rank Adaptation)
Deep Learning, Machine Learning, Natural Language Processing
LoftQ
Large Language Models
Log Loss
Machine Learning, Mathematics