ARC Evals
ARC Evals was the evaluations team incubated inside the Alignment Research Center (ARC) between 2022 and 2023, and the direct predecessor of METR (Model Evaluation and Threat Research).
AI Safety
Explore AI Safety through related topics and the articles other pages reference most.
Explore articles
Browse subtopics: Model Evaluation
Articles that also belong to these categories. Counts cover all of AI Safety.
All articles (233)People (39)AI Policy & Regulation (31)AI Alignment (26)AI Ethics (26)Machine Learning (26)AI Benchmarks (25)AI Companies (24)Artificial Intelligence (24)Large Language Models (21)Anthropic (16)Research Organizations (16)Model Evaluation (11)AI Research (9)AI Agents (7)OpenAI (7)Interpretability (6)Natural Language Processing (6)Reinforcement Learning (5)Developer Tools (4)Open Source AI (4)Robotics (4)AI Incidents & Controversies (3)AI History (2)AI Tools & Products (2)Companies (2)Computer Science (2)Generative AI (2)Google DeepMind (2)NVIDIA (2)AI Code Generation (1)AI Events (1)AI Inference (1)AI Models (1)AI in Cybersecurity (1)ChatGPT (1)Chinese AI (1)Data & Datasets (1)Data Science (1)Deep Learning (1)Defense AI (1)Finance AI (1)Google (1)Hardware (1)MLOps (1)Meta AI (1)Prompt Engineering (1)Venture Capital (1)
Showing 1-11 of 11 articles
Cybench
Cybench (short for Cybersecurity benchmark) is an open-source evaluation framework for measuring the cybersecurity capabilities and risks of large language model agents.
Distribution shift
Distribution shift is the condition in which the probability distribution that produced a model's training data differs from the distribution that produces the data the model actually encounters at test or…
NIST ARIA
NIST ARIA (Assessing Risks and Impacts of AI) is a testing, evaluation, validation, and verification (TEVV) program operated by the United States National Institute of Standards and Technology (NIST) to…
Patronus AI
Patronus AI is an automated LLM evaluation, observability, and guardrails platform founded in 2023 and headquartered in San Francisco.
Process reward model (PRM)
A process reward model (PRM), also called a process-supervised reward model or step-level verifier, is a learned scoring model that evaluates the correctness or quality of each intermediate step in a large…
Sabotage evaluations
Sabotage evaluations are a set of tests, introduced by Anthropic in October 2024, that try to measure whether a frontier language model is capable of covertly subverting human oversight, decision-making, and…
Sandbagging (artificial intelligence)
Sandbagging, in the context of AI safety, refers to the strategic and intentional underperformance of an AI system on a capability evaluation or specific task, typically to hide a capability from human…
StrongREJECT
StrongREJECT is a benchmark and automated evaluator for measuring how well jailbreaking attacks actually work against large language models.
Task-completion time horizon (METR)
The task-completion time horizon is a metric for AI capability proposed by METR that expresses a model's ability in units of human time: it is the length of task
UK AI Security Institute
The UK AI Security Institute (AISI) is a research organization within the United Kingdom's Department for Science, Innovation and Technology (DSIT) that conducts pre-deployment evaluations of frontier AI…