AI Safety

Explore AI Safety through related topics and the articles other pages reference most.

Explore articles

Browse subtopics (55)

Articles that also belong to these categories. Counts cover all of AI Safety.

Showing 121-180 of 278 articles

Gated SAE

A Gated sparse autoencoder (Gated SAE) is a sparse-autoencoder architecture for mechanistic interpretability that splits the encoder into a gating path, which decides which features are active, and a magnitude…

Machine Learning

Gemma Scope

Gemma Scope is an open, comprehensive suite of sparse autoencoders (SAEs) released by Google DeepMind in 2024 to support mechanistic interpretability research on its open-weight Gemma 2 language models.

Machine Learning

Geoffrey Irving

Geoffrey Irving is a computer scientist and artificial intelligence safety researcher who, until 2026, served as chief scientist of the UK AI Security Institute (AISI), the British government body that…

People

Gillian Hadfield

Gillian Kereldena Hadfield (born July 14, 1961) is a Canadian-American legal scholar and economist whose research sits at the intersection of law, economics, and artificial intelligence.

People

GiveWell

GiveWell is an American nonprofit charity evaluator, founded in 2007 by Holden Karnofsky and Elie Hassenfeld, that uses cost-effectiveness analysis to recommend a short list of evidence-backed global health…

Research Organizations

Gradient hacking

Gradient hacking is a hypothesised failure mode of supervised and reinforcement-learning systems in which a sufficiently capable

AI Alignment

Gray Swan AI

Gray Swan AI (also called Gray Swan; legal name Gray Swan Security Inc.) is an American AI safety and security company that tests large language models and AI agents for vulnerabilities and sells tools to…

AI CompaniesAI in Cybersecurity

Grok "MechaHitler" incident

The Grok "MechaHitler" incident was a content-moderation and AI-safety failure that occurred on July 8 to 9, 2025, when Grok, the chatbot built by xAI and integrated into the social platform X (formerly…

AI Companies

Guardrails (AI)

AI guardrails are runtime safety mechanisms that monitor, validate, and constrain the inputs and outputs of AI systems, particularly large language models (LLMs), to block harmful, inaccurate, or off-policy…

Large Language Models

HaluEval

HaluEval (Hallucination Evaluation) is a large-scale benchmark for measuring how well large language models (LLMs) can recognize hallucinated content, that is, text that conflicts with a source or cannot be…

AI BenchmarksLarge Language Models

Holden Karnofsky

Holden G. Karnofsky is an American philanthropist, nonprofit executive, and writer on artificial intelligence who co-founded the charity evaluator GiveWell (2007) and the grantmaking organization Open…

People

Humanity's Last Exam

Humanity's Last Exam (HLE) is a multi-modal AI benchmark of 2,500 public expert-level academic questions (plus a private held-out set) spanning more than 100 disciplines

AI Benchmarks

Ian Hogarth

Ian Hogarth is a British technology entrepreneur, investor, and writer who serves as the founding chair of the UK AI Safety Institute

People

Igor Babuschkin

Igor Babuschkin is a German artificial intelligence researcher and engineer who is a co-founder of xAI, the AI company that Elon Musk started in 2023.

People

Indirect prompt injection

Indirect prompt injection is a class of attack against large language model-integrated applications in which the malicious instructions that subvert the model are not supplied by the user, but are smuggled…

Large Language Models

Inner alignment

Inner alignment is the AI-safety problem of ensuring that a learned model which is itself an optimizer (a mesa-optimizer) pursues the objective the training process actually selected for (the base objective)

AI Alignment

Inspect AI

Inspect AI (usually called simply Inspect) is an open-source Python framework for evaluating large language models and AI agents, created by the UK AI Security Institute (AISI) and now developed jointly by the…

Developer ToolsModel Evaluation

Instrumental convergence

Instrumental convergence is a hypothesis in AI safety holding that a wide range of sufficiently capable agents, when pursuing almost any final goal, will rationally adopt a common set of intermediate subgoals…

AI Alignment

J. Zico Kolter

J. Zico Kolter (full name Jeremy Zico Kolter) is an American computer scientist who is a professor and head of the Machine Learning Department at Carnegie Mellon University.

People

Joe Biden

Joseph Robinette Biden Jr. is an American politician who served as the 46th President of the United States, sworn in on January 20, 2021 and leaving office on January 20, 2025.

AI Policy & RegulationPeople

JumpReLU SAE

A JumpReLU sparse autoencoder (JumpReLU SAE) is a variant of the sparse autoencoder used in mechanistic interpretability whose encoder applies a learnable per-feature threshold that forces a feature to exactly…

Machine Learning

Kate Crawford

Kate Crawford (born 1974) is an Australian scholar, author, and artist known as one of the leading critical voices on the politics of artificial intelligence.

People

LatchBio

LatchBio is an American software company in San Francisco that builds cloud infrastructure and AI agents for analyzing biological data

AI BenchmarksAI Companies

LawZero

LawZero is a nonprofit AI safety research organization founded by Turing Award-winning computer scientist Yoshua Bengio and publicly launched on June 3, 2025.

Research Organizations

Leopold Aschenbrenner

Leopold Aschenbrenner is a German-born artificial-intelligence researcher and investor best known for the June 2024 essay series Situational Awareness: The Decade Ahead and for founding the AI-focused hedge…

People

LessWrong

LessWrong is a community blog and online forum focused on human rationality, cognitive biases, decision theory, the philosophy of science, and AI safety.

Lilian Weng

Lilian Weng is an artificial intelligence researcher known for her work at OpenAI, where she spent about seven years and led the Safety Systems team as Vice President of Research and Safety, and for her…

Machine LearningPeople

LongFact / SAFE

LongFact and SAFE are a paired benchmark and evaluation method for measuring the long-form factuality of large language models, introduced by researchers at Google DeepMind and Stanford University in the 2024…

AI Benchmarks

Longtermism

Longtermism is the ethical view that positively influencing the long-term future is a key moral priority of our time. The Oxford philosopher William MacAskill proposed the term in October 2017.

AI Ethics

MASK

MASK (Model Alignment between Statements and Knowledge) is an AI safety benchmark that measures the honesty of large language models (LLMs) by testing whether a model will knowingly assert something it…

AI Benchmarks

METR

METR (Model Evaluation and Threat Research) is a nonprofit research organization based in Berkeley, California, that develops scientific methods for measuring the autonomous capabilities of frontier AI systems…

AI BenchmarksResearch Organizations

MIT "GenAI Divide" report (2025)

The "GenAI Divide" report, formally titled The GenAI Divide: State of AI in Business 2025, is a research report published in July 2025 by MIT NANDA, an initiative based at the MIT Media Lab.

AI Companies

Many-shot jailbreaking

Many-shot jailbreaking is a technique for bypassing the safety training of a large language model by filling its context window with a long series of faux dialogue turns in which an AI assistant complies with…

AnthropicLarge Language Models

Marius Hobbhahn

Marius Hobbhahn is a German AI safety researcher and the co-founder and chief executive officer of Apollo Research, a London-based organization that studies and evaluates deceptive behavior, often called…

People

Max Tegmark

Max Tegmark is a Swedish-American physicist and artificial intelligence researcher who is a professor of physics at the Massachusetts Institute of Technology (MIT) and the co-founder and chair of the Future of…

AI EthicsPeople

Mechanistic interpretability

Mechanistic interpretability (often abbreviated as mech interp or MI) is the field that reverse-engineers the internal computations of neural networks, particularly transformers

Interpretability

MedHELM

MedHELM (Holistic Evaluation of Large Language Models for Medical Tasks) is a benchmark and evaluation framework that measures how well large language models perform on realistic clinical work.

AI Benchmarks

Membership Inference Attack

A Membership Inference Attack (MIA) is a privacy attack against a trained machine learning model in which an adversary, given a candidate data record and access to the model

Machine Learning

Meredith Whittaker

Meredith Whittaker is an American technologist, researcher, and privacy advocate who serves as president of the Signal Foundation, the nonprofit behind the encrypted messaging app Signal

AI EthicsPeople

Mesa-optimization

Mesa-optimization is the situation in AI alignment research in which a learned model, typically a neural network produced by a machine-learning training process, is itself an optimizer that internally searches…

AI Alignment

Miles Brundage

Miles Brundage is an American AI policy and governance researcher best known for leading policy research at OpenAI and serving as the company's senior advisor for AGI Readiness before leaving in October 2024.

AI Policy & RegulationPeople

Model Spec

The Model Spec is a public document published by openai that defines the intended behavior of the company's language models: how they should follow instructions, when they should refuse a request, how to…

AI AlignmentOpenAI

Model collapse

Model collapse is a degenerative process in which generative AI models trained recursively on data produced by previous-generation models progressively lose information, especially the rare events in the tails…

Machine Learning