KTO
KTO (Kahneman-Tversky Optimization) is a method for aligning large language models with human feedback using only a binary signal of whether a model output is desirable or undesirable, rather than the paired…
AI Alignment
Explore AI Alignment through related topics and the articles other pages reference most.
Explore articles
Browse subtopics: Reinforcement Learning
Articles that also belong to these categories. Counts cover all of AI Alignment.
All articles (40)AI Safety (26)Training & Optimization (7)Anthropic (6)Large Language Models (5)Reinforcement Learning (5)OpenAI (4)AI Research (3)Machine Learning (3)AI Companies (2)AI Ethics (2)AI Inference (2)Deep Learning (2)Natural Language Processing (2)AI Agents (1)AI Benchmarks (1)Conversational AI (1)Google DeepMind (1)People (1)Statistics (1)
Showing 1-5 of 5 articles
RLOO (REINFORCE Leave-One-Out)
RLOO (REINFORCE Leave-One-Out) is an online reinforcement learning algorithm for aligning large language models with reward signals such as those derived from human preferences.
Recursive reward modeling
Recursive reward modeling (RRM) is a proposed approach to the scalable oversight problem in AI alignment
Reward hacking
Reward hacking (also called specification gaming) is a failure mode in artificial intelligence in which a system maximizes its given objective or reward signal through unintended shortcuts, exploits, or…
Specification gaming
Specification gaming is the phenomenon in which an optimizer satisfies the literal specification of an objective without producing the outcome that the designer actually wanted.