UK AI Security Institute

The UK AI Security Institute (AISI) is a UK government research organization (part of the Department for Science, Innovation and Technology until July 2026, when responsibility for it moved to the Cabinet Office^[40]) that conducts pre-deployment evaluations of frontier AI models and produces safety and security research to inform government policy.^[2] It is widely regarded as the world's first government body dedicated to the systematic technical evaluation of advanced AI systems.^[28]

Originally established as the UK AI Safety Institute on 2 November 2023, following Prime Minister Rishi Sunak's announcement at the AI Safety Summit held at Bletchley Park, the organization was renamed the UK AI Security Institute on 14 February 2025, when Technology Secretary Peter Kyle announced a strategic refocusing toward serious AI risks with national security implications.^[1][4] The institute's operational home page is at aisi.gov.uk, and it retains the AISI abbreviation under the new name.^[6]

As of 2026, the institute employs more than 100 technical staff drawn from industry, academia, and civil society organizations.^[6] It operates with annual funding of approximately £66 million, has access to more than £1.5 billion in computing resources through the national AI Research Resource, and has tested more than 30 of the world's most advanced AI models since its founding.^[6][8] The institute is a founding member of the International Network of AI Safety Institutes, launched with nine countries and the European Union under an inaugural US chair, and since renamed the International Network for Advanced AI Measurement, Evaluation and Science.^[43][8]

How did the AISI begin? The Frontier AI Taskforce

The institution that became the UK AI Security Institute traces its direct origins to April 2023, when the UK government announced the formation of the Frontier AI Taskforce.^[2] At the time, the UK government committed an initial £100 million to fund the organization.^[2] The stated purpose of the Taskforce was to understand the capabilities and risks of frontier AI systems before more comprehensive regulatory structures could be established.^[2]

Ian Hogarth, a technology investor and entrepreneur best known for his 2023 essay "We must slow down the race to God-like AI," was appointed to chair the Taskforce.^[2] The government's case for the body, as Sunak put it in November 2023, was that "until now, the only people testing the safety of new AI models have been the very companies developing them."^[25]

The Frontier AI Taskforce began recruiting technical staff during mid-2023 and started building evaluation frameworks and relationships with leading AI developers.^[2] During this period, Prime Minister Rishi Sunak secured informal commitments from Sam Altman of OpenAI, Demis Hassabis of Google DeepMind, and Dario Amodei of Anthropic to provide early or priority access to their models for research and safety testing.^[28] These voluntary commitments preceded any formal agreements and reflected the close dialogue the UK government had cultivated with AI company leadership during 2023.^[28]

The AI Safety Summit and AISI launch

The AI Safety Summit, held at Bletchley Park on 1 and 2 November 2023, was the first major intergovernmental conference focused specifically on the risks of frontier AI.^[3] The choice of Bletchley Park was deliberate: the site is famous for codebreaking work during the Second World War and holds symbolic resonance in British technology history.

The summit brought together delegations from 28 countries and the European Union, including the United States, China, Australia, Canada, France, Germany, India, Italy, Japan, and South Korea.^[3] On the first day of the summit, representatives from these countries and the EU signed the Bletchley Declaration, a joint statement affirming that AI should be developed and deployed in a manner that is safe, human-centric, trustworthy, and responsible.^[3] The Declaration acknowledged that frontier AI carries risks of intentional misuse, unintended issues of control, and potential for "serious, even catastrophic, harm," and committed signatory nations to work together to understand those risks.^[3]

The Declaration said developers of frontier AI have "a particularly strong responsibility" for safety, "including through systems for safety testing, through evaluations," and called for developing relevant public sector capability.^[3] In a separate session on 2 November, ten AI companies (Amazon Web Services, Anthropic, Google, Google DeepMind, Inflection AI, Meta, Microsoft, Mistral AI, OpenAI and xAI) joined like-minded governments in agreeing to collaborate on testing next-generation models against national security, safety and societal risks.^[35]

On 2 November 2023, in his closing address to the summit, Prime Minister Rishi Sunak announced the formal launch of the UK AI Safety Institute as the successor to the Frontier AI Taskforce.^[1] Ian Hogarth remained in the Chair role.^[1] The AISI was, as its own website puts it, designed "like a startup in the government": combining governmental authority and access to official information with the technical depth and operational speed more typical of a well-funded technology organization.^[6] The institute was positioned not as a regulator (it had no enforcement powers) but as an evaluation and research body whose findings would inform both UK policy and the broader international debate on AI governance.^[2]

What does the UK AI Security Institute do?

The institute's mission, as stated on its official website, is to build "the world's leading understanding of advanced AI risks and solutions, to inform governments so they can keep the public safe."^[6]

Since its founding, the AISI has organized its activities around three interconnected functions.^[6]

Testing AI systems. The institute conducts pre-release evaluations of leading AI models in collaboration with the companies that develop them.^[6] These evaluations examine how capable a model is in domains that pose safety or security concerns, whether the model's safeguards against misuse hold up against adversarial attempts to circumvent them, and whether autonomous or agentic uses of the model could produce unintended harmful outcomes.^[6]

Research advancement. The institute conducts and funds in-house research on AI safety and security topics that it considers underdeveloped in the academic literature.^[7] This includes work on evaluation methodology, model alignment, control mechanisms for autonomous AI systems, and understanding how AI capabilities in sensitive domains are evolving over time.^[7] The institute says it is mobilizing more than £15 million in grant funding for external research.^[6]

Policy guidance. The institute provides technical briefings, reports, and advice to UK government departments and to allied governments participating in the international network.^[6] Its findings on model capabilities, safeguard robustness, and emerging risks feed directly into policy discussions about how to regulate and deploy frontier AI systems.^[6]

Leadership

Ian Hogarth served as Chair of the organization from its inception as the Frontier AI Taskforce through at least 2025.^[6] Hogarth brought to the role both his public profile as an advocate for cautious AI development and his experience as a founder and investor in the technology sector.

Geoffrey Irving served as Chief Scientist until 2026, when he left to co-found the research organization Resolution.^[42] Over his machine learning career he co-led the N2Formal neural network theorem proving team at Google Brain, led the Reflection Team at OpenAI working on AGI safety and language model alignment, and subsequently led the Scalable Alignment Team at Google DeepMind.^[42] He is best known academically for developing the "debate" approach to aligning AI systems with human values, a scalable alignment technique in which two AI systems argue opposing positions and human judges evaluate the quality of their reasoning.

Chris Summerfield serves as Research Director.^[6] Summerfield is a Professor of Cognitive Neuroscience at the University of Oxford and brings expertise in how biological intelligence relates to artificial systems. His academic background informs the institute's approaches to evaluating AI reasoning, persuasion, and influence capabilities.

Jade Leung serves as Chief Technology Officer and also holds the role of Prime Minister's AI Adviser.^[6] Leung previously led Governance at OpenAI and brings deep expertise in the policy and operational dimensions of responsible AI deployment.

Adam Beaumont became Interim Director in September 2025 (the appointment was reported in October), succeeding founding director Oliver Ilott.^[30] Beaumont previously served as Chief AI Officer at GCHQ, the UK government's intelligence and cybersecurity agency.^[30] His appointment reflected the institute's increased focus on national security applications following the February 2025 renaming, and it deepened the institute's working relationship with the broader UK security and intelligence community.^[30] In August 2026 Beaumont returned to GCHQ and Henry de Zoete, a former Prime Minister's AI Adviser who helped set up the institute, was named Director, with Nate Burnikell as Chief Strategy Officer.^[41][6]

Yoshua Bengio, one of the founding researchers of modern deep learning, was commissioned by the UK government in 2023 to chair the international "State of the Science" report on advanced AI,^[2] and he chaired its February 2026 edition, the International AI Safety Report 2026, for which the institute co-provided the secretariat.^[22]

Pre-deployment evaluations

The pre-deployment evaluation program is the institute's most operationally distinctive activity.^[2] It involves the institute receiving access to an AI model before the model is released publicly, conducting structured tests to assess its capabilities and vulnerabilities, and producing findings that inform both the developing company and the UK government.^[2]

The original voluntary commitments made at and around the Bletchley Summit were subsequently formalized through memoranda of understanding with major AI developers.^[26] Memoranda of understanding covering Google DeepMind, and UK government MOUs with Anthropic, OpenAI and Cohere, include commitments to work with the institute.^[8][16] These agreements give the institute early access to models under defined conditions and establish expectations for information sharing.^[16]

The institute's May 2024 evaluation update described models only by anonymized labels (Red, Purple, Green, Blue, Yellow) rather than naming specific products,^[11] but later published evaluations, including the joint pre-deployment test of OpenAI's o1 and the 2026 Claude Mythos Preview assessment, named the models tested.^[38][10]

Methodology

Evaluations are structured around four primary risk domains: cybersecurity, chemical and biological misuse, autonomous AI behavior, and safeguard robustness.^[11]

Within each domain, the institute tests three dimensions of model behavior. Compliance measures whether the model declines to assist with requests that fall within the risk category. Correctness assesses the accuracy and practical utility of any assistance the model provides. Completion measures how successfully the model can carry a potentially harmful task through to a functional result.^[11]

Evaluators use both automated methods and expert human review.^[11] Automated evaluation allows systematic testing at scale across large numbers of prompts and scenarios. Human expert validation is used for more complex cases, particularly in domains like cybersecurity and biology, where assessing whether model output actually provides meaningful uplift to a malicious actor requires specialist judgment.^[11]

The institute also employs adversarial red-teaming to assess safeguards.^[7] Rather than accepting model refusals at face value, red-teamers attempt to circumvent the safeguard through jailbreak techniques, indirect prompting, or by exploiting gaps between what a model is trained to decline and what it can be induced to do with creative framing.^[7]

Key findings

The institute's May 2024 advanced evaluations update, its first technical blog post of results, covering five publicly available models, found that the most capable models evaluated could solve more than half of capture-the-flag cybersecurity challenges designed for high school students but struggled with university-level challenges.^[11] In the life sciences domain, several models matched PhD-level performance on expert-written questions covering basic biology, advanced biological topics, and laboratory automation.^[11]

On autonomous capabilities, models evaluated in 2024 completed 20 to 40 percent of short-horizon software engineering tasks but failed all long-horizon tasks, hampered by syntax errors and insufficient self-testing.^[11]

All models evaluated in the 2024 report remained highly vulnerable to basic jailbreaks, and even without an attack one model complied with up to 28 percent of private harmful questions.^[11]

Frontier AI Trends Report

In 2025, the institute published its inaugural Frontier AI Trends Report, drawing on two years of pre-deployment evaluations to characterize the trajectory of AI capability development across the six primary domains it monitors: chemistry and biology, cybersecurity, autonomous capabilities, safeguard robustness, loss-of-control risks, and societal impact.^[9]

The report's headline findings included a doubling of performance in some capability domains approximately every eight months.^[9] On cybersecurity tasks, models that completed apprentice-level tasks less than 9 percent of the time in late 2023 were completing them 50 percent of the time on average by the report's publication in December 2025.^[9] In the life sciences domain, frontier models had surpassed the performance of human PhD experts on open-ended biology questions, and were generating detailed experimental protocols tailored to the recipient's level of knowledge.^[9]

Safeguard analysis in the Trends Report found that universal jailbreaks (techniques that cause every tested system to comply with harmful requests) existed in every system the institute had evaluated.^[9] However, the report also found that the amount of expert time required to discover effective jailbreaks was increasing for certain models and request categories, indicating that safeguard robustness was improving even if absolute robustness remained elusive.^[9]

On autonomous AI and loss-of-control risks, the report documented that self-replication success rates for models attempting to copy themselves across compute environments had increased from approximately five percent in 2023 to sixty percent by 2025.^[9] The institute reported no evidence of spontaneous sandbagging (strategic underperformance by models in evaluation settings) but noted it had developed specific detection methods for this behavior given its implications for the reliability of safety evaluations.^[9]

Claude Mythos evaluation (2026)

Among the institute's most significant published evaluations was its 2026 assessment of Anthropic's Claude Mythos Preview model.^[10] The evaluation focused on cybersecurity capabilities and found that Claude Mythos Preview succeeded on 73 percent of expert-level capture-the-flag tasks, a level no model could complete at all before April 2025.^[10] More significantly, it was the first model to complete "The Last Ones" simulation from start to finish in three out of ten attempts.^[10] The Last Ones is a 32-step simulation of a corporate network attack estimated to require humans twenty hours to complete.^[10]

The institute concluded that Claude Mythos Preview could autonomously attack small, weakly defended, and vulnerable enterprise systems when given network access.^[10] It recommended that organizations prioritize cybersecurity fundamentals such as regular security updates, robust access controls, security configuration and comprehensive logging, while noting that its test environments lacked active defenders and real-world security tooling, meaning results might not directly extrapolate to hardened systems.^[10] Anthropic separately stated that Mythos Preview had found thousands of high-severity vulnerabilities, "including some in every major operating system and web browser."^[36] The institute noted that AI cyber capabilities are dual use, able to strengthen defence as well as attack.^[10]

Fraud and cybercrime evaluation framework (February 2026)

In February 2026, the institute published an evaluation framework specifically targeting AI misuse in fraud and cybercrime, addressing what it described as a rapidly growing real-world threat.^[14] Background incidents cited by the institute included criminals using large language models to secure remote employment under false identities, to profile victims, and to craft sophisticated phishing campaigns.^[14]

The framework's methodological novelty was a battery of long-form tasks (LFTs) that mirror how fraud and cybercrime unfold in practice.^[14] Unlike most existing evaluations, which focus on single-turn prompts, LFTs simulate the multi-step interactions that real malicious actors use.^[14] The institute built them around three scenarios (romance scams, CEO impersonation and identity theft), each with three adversary profiles (non-technical individuals, technically skilled individuals and organised groups), and ran more than 20,000 evaluations across 14 text-based models, scoring responses for actionability and information access with rubrics developed with law-enforcement experts.^[14]

The study found that the text-based models tested provided limited practical assistance for fraud: 88.5 percent of responses scored low on actionability, 67.5 percent scored low on information access, and only 5 to 7 percent produced material that looked ready to use.^[14] Safety alignment, not capability, determined misuse risk: there was no similar trend across model size, newer model versions, or capabilities such as extended reasoning and web search.^[14] Closed-weight, safety-aligned models accessed through commercial APIs consistently refused harmful requests across the LFT suite.^[14] Misuse risk concentrated almost entirely in open-weight models that had been fine-tuned to remove their safety guardrails ("uncensored" models, which the institute noted are widely available on open-weight platforms).^[14]

The institute concluded that the results highlighted the importance of deployment safeguards, alignment standards and active monitoring of open-weight models.^[14] As of August 2026, the UK still had no cross-sector AI statute and no rule aimed specifically at open-weight models whose safeguards have been stripped, governing AI instead through existing sectoral regulators and the non-binding principles set out in the October 2025 DSIT blueprint.^[34]

What is the Inspect evaluation framework?

In May 2024, the institute released Inspect, an open-source Python framework for conducting large language model evaluations.^[12] The release made public the core software infrastructure underlying the institute's own evaluation practice and positioned the institute as a contributor to the broader ecosystem of AI evaluation tooling.^[12]

Inspect is installable via a single pip command and is released under the MIT license, making it freely available for commercial and non-commercial use.^[12] The framework supports evaluations covering reasoning, coding, knowledge assessment, agentic task completion, behavior analysis, and multi-modal understanding.^[12] It includes a collection of more than 200 pre-built evaluations that can be run against any supported model.^[12]

The framework provides a web-based visualization tool called Inspect View for monitoring and reviewing evaluation runs, as well as a VS Code extension for authoring and debugging evaluations.^[12] It supports flexible tool-calling capabilities including custom tools, MCP tools, and built-in utilities for bash execution, code interpretation, web search, and web browsing.^[12]

The institute subsequently expanded the Inspect ecosystem with InspectSandbox, a secure execution environment for agentic evaluations; InspectCyber, a specialized collection of cybersecurity evaluation tasks; and ControlArena, a library specifically designed for AI control experiments.^[8][13] By December 2025 the institute said these tools were being used by governments, companies and academics around the world.^[8]

ControlArena

ControlArena, released in 2025, is an open-source research platform for studying whether safety protocols can prevent advanced AI models from taking actions that circumvent human oversight.^[13] It provides standardized environments in which researchers can test specific control techniques against AI systems modeled on frontier developer infrastructure.^[13]

The institute's research agenda says its control work, tested with ControlArena, focuses on four side-tasks mapped to core threat models: weight exfiltration (whether an AI model can copy its own parameters to an external location), secret exfiltration (whether a model can leak confidential information), rogue internal deployment (whether a model can spin up new instances of itself), and sabotage of safety research (whether a model can subtly corrupt the evaluation process it is being subjected to).^[7]

The platform has been used by researchers at Anthropic, Google DeepMind, Redwood Research, and other organizations.^[13] Its release reflects the institute's broader strategy of producing public goods for the AI safety research community rather than keeping evaluation infrastructure proprietary.^[13]

Research agenda

The institute organizes its research program into two broad categories: risk research, which characterizes the severity and trajectory of AI-related threats, and solutions research, which develops and tests approaches to mitigating those threats.^[7]

Risk research

Domain-specific risk research covers six areas: cyber misuse, dual-use science (particularly in chemistry and biology), criminal misuse (including AI-enabled fraud and child sexual abuse material), autonomous AI systems (including loss-of-control and unintended harmful action), societal resilience (risks from widespread AI deployment affecting critical infrastructure or social cohesion), and human influence and manipulation.^[7]

In parallel, the institute conducts generalized research on the science of evaluations, developing more rigorous statistical and methodological frameworks for measuring AI capabilities, and on capability elicitation, the study of how to accurately surface the true performance ceiling of a model rather than its typical behavior under standard prompting.^[7]

Solutions research

Safeguard analysis examines whether defensive measures deployed by AI developers actually hold up under adversarial conditions.^[7] The institute uses adversarial machine learning techniques to probe whether model safeguards can be circumvented and characterizes the difficulty of finding effective jailbreaks as a proxy for safeguard robustness.^[7]

Control research develops protocols for preventing AI models from taking actions that circumvent human oversight even in the context of advanced AI systems that may have strategic capabilities.^[7] The institute uses mock environments replicating frontier developer infrastructure to test whether proposed control protocols would succeed against capable adversarial models.^[7]

Alignment research pursues approaches to ensuring AI systems behave in accordance with human values and intentions, including work on detecting deceptive AI behavior and establishing formal guarantees around model honesty.^[7] The institute's Alignment Project, launched in July 2025 with more than £15 million in initial backing from an international coalition, is one of the largest dedicated alignment research efforts globally.^[33][8] By its first funding round, the project had received more than 800 applications from 466 institutions across 42 countries, and awarded its first 60 grants.^[33] New partners including OpenAI (which contributed £5.6 million) and Microsoft brought the total committed to alignment research to £27 million.^[33]

The institute also conducts empirical persuasion research.^[8] In 2025, it published a large-scale study involving more than 76,000 participants examining the mechanisms through which AI-generated content influences human attitudes, a study the institute framed as groundwork for understanding AI-driven manipulation risks at societal scale.^[8]

Why was the AI Safety Institute renamed the AI Security Institute? (February 2025)

On 14 February 2025, Technology Secretary Peter Kyle announced at the Munich Security Conference that the UK AI Safety Institute would be renamed the UK AI Security Institute.^[5] The announcement came three days after the AI Action Summit in Paris, which had convened under French presidency to advance the post-Bletchley agenda on AI governance.^[4]

Kyle framed the renaming as an alignment of the institute's title with its actual activities.^[5] "I am renaming our AI Safety Institute as the AI Security Institute," Kyle told the conference.^[5] In a written ministerial statement he added: "This change of name better reflects the team's essential remit, and the priorities they have always focussed on."^[31] In the accompanying press release he described the broader package as "the logical next step in how we approach responsible AI development, helping us to unleash AI and grow the economy as part of our Plan for Change."^[4] He cast the most pressing AI risks as those with direct security implications: cyber attacks, the potential for AI to assist in the development of chemical or biological weapons, criminal misuse including fraud and child sexual exploitation material, and risks to critical national infrastructure.^[4]

The renaming was accompanied by structural changes.^[4] A new criminal misuse team was created within the institute, partnering with the Home Office to research AI-enabled crime.^[4] The institute was tasked with taking "a leading role in testing AI models wherever they come from, open or closed."^[31] A new agreement with Anthropic was announced, described as part of a broader Sovereign AI initiative.^[26]

The February 2025 announcement explicitly stated that the institute would no longer prioritize work on AI ethical issues such as algorithmic bias or freedom of expression in AI applications, marking a deliberate narrowing of scope compared to the institute's original mandate.^[4]

In the same announcement, the institute confirmed an expanded set of working relationships with other parts of the UK security architecture.^[4] The institute would partner with the Defence Science and Technology Laboratory, the Laboratory for AI Security Research (LASR) and the national security community, including building on the expertise of the National Cyber Security Centre (NCSC).^[4] A written ministerial statement on 24 February 2025 repeated the new remit and said the institute's research would be shared with allies.^[31]

Reactions to the renaming

The renaming generated significant commentary from AI governance researchers and civil society organizations.

The AI Now Institute issued a statement warning that the shift from safety to security framing risked applying "piecemeal or superficial scrutiny" to AI systems before they were ready for deployment in defense and national security applications.^[23] AI Now argued that frontier AI models carry cyber vulnerabilities that themselves pose national security risks, and that independent safety evaluation must be insulated from industry partnerships to retain credibility.^[23]

Others interpreted the renaming more pragmatically; the institute's chair, Ian Hogarth, said its "focus from the start has been on security".^[29] Its actual evaluation work had always included substantial security-relevant content. Its cybersecurity evaluations, CBRN misuse assessments, and safeguard robustness testing all had direct security applications, and the name change formalized an existing emphasis rather than fundamentally redirecting the organization.^[29]

The government's written statement of 24 February 2025 said bias and freedom-of-speech issues would be handled elsewhere in DSIT, through the Fairness Innovation Challenge and its AI assurance work.^[31]

Government and industry agreements

The institute's ability to conduct pre-deployment evaluations depends on voluntary agreements with AI developers, since the UK has not yet enacted legislation requiring mandatory pre-deployment testing.^[25]

Anthropic. The AISI and Anthropic have maintained one of the most extensively documented evaluation partnerships.^[26] This relationship produced the institute's assessments of Claude 3.5 Sonnet, which Anthropic provided to UK AISI for pre-deployment evaluation in June 2024,^[44] and the 2026 Claude Mythos Preview evaluation.^[10] Anthropic and the AISI also collaborated on biosecurity red-teaming that identified dozens of vulnerabilities including new universal jailbreak paths, and on the largest backdoor data poisoning study conducted to date. A memorandum of understanding between the UK government and Anthropic, under which Anthropic would also provide tools to help the institute evaluate AI capabilities, was announced alongside the February 2025 renaming.^[26]

OpenAI. The UK AISI and the US AI Safety Institute (housed at NIST) conducted a joint pre-deployment evaluation of OpenAI's o1 model, published in December 2024,^[38] a month after their joint evaluation of Anthropic's upgraded Claude 3.5 Sonnet.^[39] This reflected both the maturation of the bilateral US-UK AI safety relationship and the specific interest both governments had in the capabilities of advanced reasoning models.

Google DeepMind. Google DeepMind has partnered with the AISI since the institute's inception in November 2023.^[16] The partnership deepened through a formal research partnership agreement announced in 2025, with Google DeepMind researchers also among ControlArena's users.^[13]

Microsoft. In May 2026, Microsoft signed parallel agreements with the UK AISI and the US Center for AI Standards and Innovation (CAISI), the renamed successor to the US AI Safety Institute, covering pre-deployment evaluation, joint methodology development, and information sharing on national-security-relevant capabilities.^[17] In the UK, the agreement covers collaborative research on methods for evaluating high-risk capabilities and the effectiveness of safeguards, plus societal resilience research on how conversational AI systems interact with users in sensitive contexts.^[17]

Cohere and others. Cohere's memorandum of understanding with the UK government, like those of Anthropic and OpenAI, includes commitments to work with the institute.^[8]

The UK AISI Agent Red-Teaming Challenge

In early 2025, the institute partnered with Gray Swan AI to run the UK AISI Agent Red-Teaming Challenge, described at the time as the largest public evaluation of agentic large language model safety conducted to date.^[24] The challenge ran from 8 March to 6 April 2025 and invited external researchers to attempt to find safety failures in deployed agentic AI systems.^[24]

Participants made approximately 1.8 million attempts to break agent behaviors, resulting in 62,000 successful exploits across 22 different large language models targeting 44 specific categories of harmful agentic behavior.^[24] The results provided the institute with a large dataset on the types of agent safety failures that adversarial users could reliably produce and informed subsequent research on agentic AI safeguards.^[24]

International counterparts and the global network

The UK AI Safety Institute was the first national government body of its kind when it launched in November 2023.^[28] Within eighteen months, analogous institutions had been established or announced in about a dozen jurisdictions, and the UK institute had taken a central role in coordinating their activities.^[32]

US Center for AI Standards and Innovation (CAISI). The United States established its AI Safety Institute within the National Institute of Standards and Technology (NIST) in early 2024.^[18] In 2025 the body was restructured and renamed the Center for AI Standards and Innovation, with a remit that placed greater emphasis on measurement, evaluation infrastructure, and standards-setting than the predecessor institute.^[18] CAISI inherited the bilateral relationship with the UK AISI, including the joint OpenAI o1 evaluation, and remained the UK institute's closest international counterpart through to 2026.^[18] Following a September 2026 executive order on "super intelligence" terminology, NIST renamed it the Center for Advancing Innovation and Standards for Super Intelligence (CAISSI) in October 2026.^[18][37]

Japan AI Safety Institute. Japan established its AI Safety Institute in February 2024 under the Information Technology Promotion Agency, with a staff of approximately 23 people.^[32]

Singapore AISI. In May 2024, Singapore renamed its Digital Trust Centre as the Singapore AI Safety Institute, placing it under Nanyang Technological University with S$10 million in annual funding.^[32]

South Korea. South Korea announced in May 2024 that it would create an AI safety institute under the Electronics and Telecommunications Research Institute.^[32]

EU AI Office. The European Union's AI Office, founded in May 2024 as part of the broader AI Act implementation framework, is a member of the international network.^[32] Unlike the national institutes, the EU AI Office has regulatory powers: it can request information from frontier model providers and apply sanctions to enforce the AI Act's requirements.^[32] This regulatory authority gives the EU AI Office a different character from the purely advisory and research-oriented national institutes.

The International Network of AI Safety Institutes

At the AI Seoul Summit in May 2024, world leaders from Australia, Canada, the European Union, France, Germany, Italy, Japan, the Republic of Korea, Singapore, the United Kingdom, and the United States signed the Seoul Statement of Intent toward International Cooperation on AI Safety Science, formally launching the International Network of AI Safety Institutes.^[19]

The network's stated objectives include accelerating the science of AI safety, promoting complementarity and interoperability among national evaluation methodologies, and fostering a common international understanding of AI safety approaches.^[21] The UK AISI has served as a founding anchor of the network, contributing evaluation methodology, the open-source Inspect framework, and bilateral research agreements as key inputs to the collective effort.^[20]

The network held its first in-person convening in San Francisco in November 2024, attended by representatives from Australia, Canada, the European Commission, France, Japan, Kenya, South Korea, Singapore, the UK, and the United States.^[21] The convening reported the network's first joint testing exercise, on Meta's Llama 3.1 405B, led by technical experts from the US, UK, and Singapore institutes.^[43]

By late 2025 the network, which the institute had helped launch, had evolved into the International Network for Advanced AI Measurement, Evaluation and Science, a change the institute said "reflects a shared commitment to rigorous, evidence-driven approaches to understanding advanced AI."^[8]

International AI Safety Report 2026

On 3 February 2026, the second edition of the International AI Safety Report was published, with the AI Security Institute and Mila - Quebec AI Institute providing its secretariat.^[22] The report was led by Yoshua Bengio as its Chair, was authored by more than 100 AI experts, and was backed by over 30 countries and international organizations; its publishers called it the largest global collaboration on AI safety to date.^[22]

The report describes itself as building on the mandate given by world leaders at the 2023 Bletchley Park summit to produce an evidence base for critical decisions about general-purpose AI.^[22]

How does the AISI differ from other AI evaluation bodies?

The UK institute is distinguished from purely academic or civil society evaluation bodies by its governmental status, which gives it privileged access to security and intelligence information and official standing in international diplomacy.^[28] It is distinguished from the EU AI Office by the absence of regulatory powers: the institute cannot compel AI developers to participate in evaluations or to modify their systems in response to findings.^[32]

Independent evaluation organizations such as Apollo Research and METR operate in adjacent spaces. METR (formerly ARC Evals) specializes in evaluating autonomous AI capabilities and has conducted pre-deployment evaluations of models from Anthropic and others, sometimes in coordination with the AISI. Apollo Research focuses on the question of whether AI systems exhibit deceptive or manipulative behaviors toward humans, a research question that overlaps substantially with the AISI's alignment and control research agenda.

Reception and assessment

The institute has received broadly favorable assessments from AI governance researchers who view independent governmental evaluation capability as a necessary component of responsible AI development at scale.^[28]

The voluntary nature of the evaluation agreements has been a persistent subject of scrutiny.^[27] Critics have noted that AI developers retain significant control over what they share with the institute, when they share it, and how evaluation findings are communicated publicly.^[27] Time reported in January 2025 that the institute often does not publicly disclose the results of its evaluations, citing security and intellectual-property reasons, and its May 2024 update identified models only by anonymized labels.^[28][11]

The non-binding character of the institute's guidance has also been noted.^[27] Evaluation findings inform but do not legally constrain AI developers. A company that receives an evaluation indicating significant safeguard vulnerabilities is not required to delay release or implement specific mitigations. The institute's influence operates through reputational pressure, government policy channels, and the voluntary commitments developers have made to participate in good faith.^[27]

Methodological standardization is another open question.^[27] The IAPS analysis of first-wave AI safety institutes described evaluations as "the stepping stone to standardized protocols and benchmarks," a process still under development.^[27] The Inspect framework represents a partial response to this concern.^[12] By open-sourcing its evaluation infrastructure, the institute has made it easier for others to use similar methodologies, but does not resolve the deeper question of what constitutes a definitive assessment that a given model is or is not safe.^[12]

The February 2025 renaming raised concerns among some researchers that the AISI's narrowed focus on security-oriented risks would leave other important categories of AI harm, including algorithmic bias, labor displacement, democratic manipulation, and risks to freedom of expression, without a dedicated governmental evaluation body in the UK.^[23]

The institute's 2025 year-in-review highlighted the scale of its research activities: 30-plus model evaluations, a large-scale persuasion study with more than 76,000 participants, a major agent red-teaming challenge identifying 62,000 vulnerabilities, and the publication of the Frontier AI Trends Report drawing on the institute's full two-year evaluation dataset.^[8] The 2026 publication slate (the long-form-task fraud framework, the Claude Mythos cybersecurity assessment, and the institute's contribution to the International AI Safety Report 2026) extended that record into a third year of sustained output.

How is the AISI funded?

The institute operates with funding of approximately £66 million per financial year.^[6] This follows the £100 million initial commitment made for the Frontier AI Taskforce, which the government said would be continued as an annual amount for the rest of the decade.^[2]

Beyond direct budget allocation, the institute has access to more than £1.5 billion in high-performance computing resources through the national AI Research Resource, including priority access to the Isambard-AI supercomputer.^[8] This compute access enables the institute to conduct its own model training and evaluation experiments at scales that would otherwise be cost-prohibitive for a government research body.

The institute has distributed external research grants through the Alignment Project, which grew to £27 million in total committed funding by its first awards round in early 2026, alongside an additional £8 million through Systemic Safety Grants and a £5 million Challenge Fund.^[8][33] These grant programs are designed to fund alignment and safety research at universities and independent research organizations that the institute considers strategically important but that fall outside its own core operational priorities.

Timeline of key milestones

Continue reading

• AAI safetyAI safety is the research and practice of preventing or reducing unacceptable harm from artificial intelligence systems.Mentioned 36 times above
• AnthropicAnthropic is an American artificial intelligence (AI) safety and research company founded in 2021 by Dario Amodei, Daniela Amodei, and other former OpenAI researchers, best known for the Claude family of large…Mentioned 17 times above
• OpenAIOpenAI is an American artificial intelligence research and deployment organization.Mentioned 12 times above
• Google DeepMindGoogle DeepMind is an artificial intelligence research and model-development organization at Google.Mentioned 9 times above
• AAI Safety InstitutesAI Safety Institutes are government-established organizations that test, evaluate, and research the safety and security risks of advanced artificial intelligence systemsMentioned 5 times above
• Apollo ResearchApollo Research is a technical AI safety organization founded in May 2023 and headquartered in London, United Kingdom, with additional offices in San Francisco (opened early 2026) and WashingtonMentioned 3 times above

More from See also

• Sandbagging (artificial intelligence)
• METR
• Frontier Model Forum
• AI Alignment

More to read

• Model
• International AI Safety Report

Topics:AI Policy & RegulationAI SafetyModel Evaluation

References

1. ^1 ^2 ^3UK Government. "Prime Minister launches new AI Safety Institute." GOV.UK. 2 November 2023. gov.uk/...inister-launches-new-ai-safety-institute
2. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10 ^11UK Government. "Introducing the AI Safety Institute." GOV.UK. November 2023. gov.uk/...introducing-the-ai-safety-institute
3. ^1 ^2 ^3 ^4 ^5UK Government. "The Bletchley Declaration by Countries Attending the AI Safety Summit, 1-2 November 2023." GOV.UK. gov.uk/...g-the-ai-safety-summit-1-2-november-2023
4. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9UK Government. "Tackling AI security risks to unleash growth and deliver Plan for Change." GOV.UK. 14 February 2025. gov.uk/...leash-growth-and-deliver-plan-for-change
5. ^1 ^2 ^3Peter Kyle. "Remarks at the Munich Security Conference." GOV.UK. 14 February 2025. gov.uk/...r-kyle-at-the-munich-security-conference
6. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10 ^11 ^12 ^13 ^14 ^15 ^16UK AI Security Institute. "About." aisi.gov.uk. aisi.gov.uk/about
7. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10 ^11 ^12 ^13UK AI Security Institute. "Research Agenda." aisi.gov.uk. aisi.gov.uk/research-agenda
8. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10 ^11 ^12 ^13UK AI Security Institute. "Our 2025 Year in Review." aisi.gov.uk. aisi.gov.uk/...our-2025-year-in-review
9. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8UK AI Security Institute. "Frontier AI Trends Report." aisi.gov.uk. aisi.gov.uk/frontier-ai-trends-report
10. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9UK AI Security Institute. "Our evaluation of Claude Mythos Preview's cyber capabilities." aisi.gov.uk. aisi.gov.uk/...-mythos-previews-cyber-capabilities
11. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10UK AI Security Institute. "Advanced AI evaluations at AISI: May update." aisi.gov.uk. aisi.gov.uk/...advanced-ai-evaluations-may-update
12. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9UK AI Security Institute. "Inspect." inspect.aisi.org.uk. inspect.aisi.org.uk
13. ^1 ^2 ^3 ^4 ^5 ^6UK AI Security Institute. "Introducing ControlArena." aisi.gov.uk. aisi.gov.uk/...-for-running-ai-control-experiments
14. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10UK AI Security Institute. "An evaluation framework for AI misuse in fraud and cybercrime." aisi.gov.uk. February 2026. aisi.gov.uk/...r-ai-misuse-in-fraud-and-cybercrime
15. UK AI Security Institute. "How will AI enable the crimes of the future?" aisi.gov.uk. aisi.gov.uk/...-ai-enable-the-crimes-of-the-future
16. ^1 ^2 ^3Google DeepMind. "Deepening AI Safety Research with UK AI Security Institute." deepmind.google. deepmind.google/...th-the-uk-ai-security-institute
17. ^1 ^2Microsoft. "Advancing AI evaluation with the Center for AI Standards and Innovation (US) and the AI Security Institute (UK)." blogs.microsoft.com. 5 May 2026. blogs.microsoft.com/...he-ai-security-institute-uk
18. ^1 ^2 ^3 ^4NIST. "Center for AI Standards and Innovation (CAISI)." nist.gov. nist.gov/caisi
19. ^UK Government. "Seoul Statement of Intent toward International Cooperation on AI Safety Science." GOV.UK. 21 May 2024. gov.uk/...afety-science-ai-seoul-summit-2024-annex
20. ^UK Government. "Global leaders agree to launch first international network of AI Safety Institutes." GOV.UK. 21 May 2024. gov.uk/...-institutes-to-boost-understanding-of-ai
21. ^1 ^2NIST. "International Network of AI Safety Institutes Mission Statement." nist.gov. nist.gov/...ai-safety-institutes-mission-statement
22. ^1 ^2 ^3 ^4International AI Safety Report. "International AI Safety Report 2026." internationalaisafetyreport.org. 3 February 2026. internationalaisafetyreport.org/...ety-report-2026
23. ^1 ^2 ^3AI Now Institute. "AI Now Statement on the UK AI Safety Institute transition to the UK AI Security Institute." ainowinstitute.org. 2025. ainowinstitute.org/...the-uk-ai-security-institute
24. ^1 ^2 ^3 ^4Gray Swan AI. "UK AISI x Gray Swan Agent Red-Teaming Challenge: Results Snapshot." grayswan.ai. 2025. grayswan.ai/...-teaming-challenge-results-snapshot
25. ^1 ^2TechCrunch. "At Bletchley, Rishi Sunak confirms AI Safety Institute but delays regulations for another day." 2 November 2023. techcrunch.com/...lays-regulations-for-another-day
26. ^1 ^2 ^3 ^4TechCrunch. "UK drops 'safety' from its AI body, now called AI Security Institute, inks MOU with Anthropic." 13 February 2025. techcrunch.com/...nstitute-inks-mou-with-anthropic
27. ^1 ^2 ^3 ^4 ^5 ^6Institute for AI Policy and Strategy. "Understanding the First Wave of AI Safety Institutes: Characteristics, Functions, and Challenges." iaps.ai. iaps.ai/...understanding-aisis
28. ^1 ^2 ^3 ^4 ^5 ^6 ^7Time. "Inside the U.K.'s Bold Experiment in AI Safety." time.com. time.com/...uk-ai-safety-institute
29. ^1 ^2Computer Weekly. "Government renames AI Safety Institute and teams up with Anthropic." computerweekly.com. 2025. computerweekly.com/...-and-teams-up-with-Anthropic
30. ^1 ^2 ^3Transformer News. "UK AISI hires ex-GCHQ AI chief Adam Beaumont as interim director." transformernews.ai. 2025. transformernews.ai/...res-gchq-chief-adam-beaumont
31. ^1 ^2 ^3 ^4UK Parliament. "AI Security Institute - Written statement HLWS454." questions-statements.parliament.uk. 24 February 2025. questions-statements.parliament.uk/...hlws454
32. ^1 ^2 ^3 ^4 ^5 ^6 ^7Wikipedia. "Artificial intelligence safety institute." en.wikipedia.org. en.wikipedia.org/...AI_Safety_Institute
33. ^1 ^2 ^3 ^4UK AI Security Institute. "Funding 60 projects to advance AI alignment research." aisi.gov.uk. 2026. aisi.gov.uk/...ts-to-advance-ai-alignment-research
34. ^Glacis. "UK AI regulation: the pro-innovation approach." glacis.io. Updated August 2026. glacis.io/guide-uk-ai-regulation
35. ^UK Government. "Safety Testing: Chair's Statement of Session Outcomes, 2 November 2023." GOV.UK. 2 November 2023. gov.uk/...ment-of-session-outcomes-2-november-2023
36. ^Anthropic. "Project Glasswing." anthropic.com. April 2026. anthropic.com/glasswing
37. ^Cloud Security Alliance. "EO 14434 and CAISSI: Mostly a Rebrand, With One Provision to Watch." labs.cloudsecurityalliance.org. 7 October 2026. labs.cloudsecurityalliance.org/...-caissi-20261007
38. ^1 ^2UK AI Security Institute. "Pre-Deployment evaluation of OpenAI's o1 model." aisi.gov.uk. 18 December 2024. aisi.gov.uk/...ment-evaluation-of-openais-o1-model
39. ^UK AI Security Institute. "Pre-deployment evaluation of Anthropic's upgraded Claude 3.5 Sonnet." aisi.gov.uk. 19 November 2024. aisi.gov.uk/...thropics-upgraded-claude-3-5-sonnet
40. ^Civil Service World. "Burnham merges DBT with DSIT and gives 'digital' back to DCMS." civilserviceworld.com. 21 July 2026. civilserviceworld.com/...ives-digital-back-to-dcms
41. ^UKTN. "AI Security Institute adds new executives to top team." uktech.news. 21 August 2026. uktech.news/...new-executives-to-top-team-20260821
42. ^1 ^2LawZero. "Geoffrey Irving." lawzero.org. lawzero.org/...geoffrey-irving
43. ^1 ^2NIST. "FACT SHEET: U.S. Department of Commerce & U.S. Department of State Launch the International Network of AI Safety Institutes at Inaugural Convening in San Francisco." nist.gov. November 2024. nist.gov/...-department-state-launch-international
44. ^Anthropic. "Introducing Claude 3.5 Sonnet." anthropic.com. 20 June 2024. anthropic.com/...claude-3-5-sonnet

Show all 44 references