Scalable oversight
Scalable oversight is the AI safety problem of how humans can reliably supervise, evaluate, and provide training signal to artificial intelligence systems whose capabilities approach, equal, or exceed those of…
Explore AI Safety through related topics and the articles other pages reference most.
Articles that also belong to these categories. Counts cover all of AI Safety.
Showing 241-278 of 278 articles
Scalable oversight is the AI safety problem of how humans can reliably supervise, evaluate, and provide training signal to artificial intelligence systems whose capabilities approach, equal, or exceed those of…
The Seoul Declaration is the short name for the Seoul Declaration for Safe, Innovative and Inclusive AI, a non-binding international statement adopted on 21 May 2024 by 10 countries plus the European Union at…
ShieldGemma is a family of open safety-classifier models from Google, built on the Gemma family of lightweight open models.
Simon Willison is a software developer and writer who co-created the Django web framework, created the open source data tool Datasette and the LLM command-line tool, and coined the term "prompt injection" in…
SimpleQA is a factuality benchmark released by OpenAI on October 30, 2024 that measures whether large language models can answer short, fact-seeking questions correctly instead of producing hallucinations.
Situational Awareness: The Decade Ahead is a 165-page essay series published on June 4, 2024 by Leopold Aschenbrenner, a former member of OpenAI's Superalignment team.
"Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training" is a January 2024 Anthropic research paper (arXiv:2401.05566, submitted 10 January 2024) that deliberately trained large language…
Specification gaming is the phenomenon in which an optimizer satisfies the literal specification of an objective without producing the outcome that the designer actually wanted.
StrongREJECT is a benchmark and automated evaluator for measuring how well jailbreaking attacks actually work against large language models.
Stuart Jonathan Russell (born 1962) is a British computer scientist, professor emeritus of computer science (and Professor in the Graduate School) at the University of California, Berkeley, and one of the most…
The "Studio Ghibli ChatGPT moment" refers to a viral internet trend that erupted in late March 2025, after OpenAI released a new native image generation capability inside ChatGPT, powered by the GPT-4o model.
Superalignment is the technical problem of steering and controlling AI systems that are far more capable than their human supervisors, that is, systems at or beyond the level of superintelligence, and the name…
Superintelligence is a proposed level of intelligence that would greatly exceed the cognitive performance of the best humans in practically every field.
Sycophancy in artificial intelligence is the tendency of large language models to tell users what they want to hear: tailoring responses to match a user's perceived beliefs, preferences, or emotional state…
SynthID is a family of digital watermarking technologies developed by Google DeepMind for marking and identifying content generated by generative AI systems.
SynthID Bio is a family of watermarking methods developed by Google DeepMind for AI-designed protein sequences and predicted biomolecular structures.
A system prompt is a special set of instructions, guidelines, persona definitions, and contextual information given to a large language model (LLM) before any user input
TESCREAL is an acronym coined by the computer scientist Timnit Gebru and the philosopher Émile P. Torres for a group of seven overlapping ideologies: transhumanism, Extropianism, singularitarianism, cosmism…
The task-completion time horizon is a metric for AI capability proposed by METR that expresses a model's ability in units of human time: it is the length of task
Tesla Autopilot is the family of driver-assistance features that Tesla builds into its cars, together with the more capable add-on package that Tesla has sold since 2016 under the Full Self-Driving name and…
The Anthropic Institute is a research organization within Anthropic dedicated to studying the societal challenges posed by increasingly powerful artificial intelligence systems.
Tilly Norwood is a fully AI-generated synthetic performer, marketed as an "AI actress," created in 2025 by the Dutch comedian, actress, and producer Eline Van der Velden through her production company…
Timnit Gebru is an Ethiopian-born computer scientist and a leading researcher in AI ethics, best known for co-authoring the 2018 "Gender Shades" study on bias in facial recognition, co-leading Google's Ethical…
Toby Ord is an Australian moral philosopher at the University of Oxford who founded the effective-altruism organisation Giving What We Can in 2009 and wrote the 2020 book The Precipice: Existential Risk and…
A TopK SAE (TopK sparse autoencoder) is a variant of sparse autoencoder that enforces sparsity by keeping only the K largest latent pre-activations for each input and zeroing all the rest
ToxiGen is a large-scale, machine-generated dataset designed for adversarial and implicit hate speech detection.
Transhumanism is an intellectual and cultural movement that holds that the human condition can and should be fundamentally improved through science and technology, in particular through technologies that…
Tristan Harris is an American technology ethicist, public advocate, and entrepreneur best known as a co-founder and the president of the Center for Humane Technology
TruthfulQA is a benchmark designed to measure whether large language models (LLMs) generate truthful answers to questions.
The UK AI Security Institute (AISI) is a UK government research organization (part of the Department for Science, Innovation and Technology until July 2026, when responsibility for it moved to the Cabinet…
The US AI Safety Institute (USAISI or US AISI), since June 2025 the Center for AI Standards and Innovation (CAISI), and since October 2026 styled on NIST's website as the Center for Advancing Innovation and…
Vending-Bench is a family of benchmarks from Andon Labs that measures the long-term coherence of large language model-based AI agents by tasking them with running a simulated vending-machine business.
The Weapons of Mass Destruction Proxy (WMDP) is a publicly released benchmark and unlearning testbed for large language models, introduced in the paper "The WMDP Benchmark: Measuring and Reducing Malicious Use…
"We Must Pace the Frontier" is an essay by Dario Amodei, chief executive of Anthropic, published on his personal website on Saturday, September 12, 2026.
"What if automating AI R&D triggers an intelligence explosion?" is a 22-author working paper published on September 28
William David MacAskill (born William Crouch; 24 March 1987) is a Scottish moral philosopher, author, and a co-founder of the effective altruism movement, best known as the leading public proponent of…
The Youth AI Safety Institute is an independent research and testing organization launched by Common Sense Media on May 5, 2026, in San Francisco to evaluate the artificial intelligence products that children…
Émile P. Torres is an American philosopher and intellectual historian whose work concerns existential risk, the ethics of human extinction and the ideologies behind the push to build artificial general…