Process reward model (PRM)
A process reward model (PRM), also called a process-supervised reward model or step-level verifier, is a learned scoring model that evaluates the correctness or quality of each intermediate step in a large…
AI Safety
Explore AI Safety through related topics and the articles other pages reference most.
Explore articles
Browse subtopics: Reinforcement Learning
Articles that also belong to these categories. Counts cover all of AI Safety.
All articles (233)People (39)AI Policy & Regulation (31)AI Alignment (26)AI Ethics (26)Machine Learning (26)AI Benchmarks (25)AI Companies (24)Artificial Intelligence (24)Large Language Models (21)Anthropic (16)Research Organizations (16)Model Evaluation (11)AI Research (9)AI Agents (7)OpenAI (7)Interpretability (6)Natural Language Processing (6)Reinforcement Learning (5)Developer Tools (4)Open Source AI (4)Robotics (4)AI Incidents & Controversies (3)AI History (2)AI Tools & Products (2)Companies (2)Computer Science (2)Generative AI (2)Google DeepMind (2)NVIDIA (2)AI Code Generation (1)AI Events (1)AI Inference (1)AI Models (1)AI in Cybersecurity (1)ChatGPT (1)Chinese AI (1)Data & Datasets (1)Data Science (1)Deep Learning (1)Defense AI (1)Finance AI (1)Google (1)Hardware (1)MLOps (1)Meta AI (1)Prompt Engineering (1)Venture Capital (1)
Showing 1-5 of 5 articles
RLAIF
Reinforcement Learning from AI Feedback (RLAIF) is a family of alignment techniques for large language models in which the preference labels used to fine-tune a model are produced by another AI system
Recursive reward modeling
Recursive reward modeling (RRM) is a proposed approach to the scalable oversight problem in AI alignment
Reward hacking
Reward hacking (also called specification gaming) is a failure mode in artificial intelligence in which a system maximizes its given objective or reward signal through unintended shortcuts, exploits, or…
Specification gaming
Specification gaming is the phenomenon in which an optimizer satisfies the literal specification of an objective without producing the outcome that the designer actually wanted.