Direct Preference Optimization (DPO)
Direct Preference Optimization (DPO) is a method for aligning large language models with human preferences that replaces the multi-stage reinforcement learning from human feedback (RLHF) pipeline with a single…
AI Alignment
Explore AI Alignment through related topics and the articles other pages reference most.
Explore articles
Browse subtopics: Machine Learning
Articles that also belong to these categories. Counts cover all of AI Alignment.
All articles (40)AI Safety (26)Training & Optimization (7)Anthropic (6)Large Language Models (5)Reinforcement Learning (5)OpenAI (4)AI Research (3)Machine Learning (3)AI Companies (2)AI Ethics (2)AI Inference (2)Deep Learning (2)Natural Language Processing (2)AI Agents (1)AI Benchmarks (1)Conversational AI (1)Google DeepMind (1)People (1)Statistics (1)
Showing 1-3 of 3 articles
Reinforcement Learning from Human Feedback (RLHF)
Reinforcement learning from human feedback (RLHF) is a family of methods in which judgments supplied by people are converted into a reward signal and a policy is optimized with reinforcement learning against…
Reward hacking
Reward hacking (also called specification gaming) is a failure mode in artificial intelligence in which a system maximizes its given objective or reward signal through unintended shortcuts, exploits, or…