The US AI Safety Institute (USAISI or US AISI), since June 2025 the Center for AI Standards and Innovation (CAISI), is a research and evaluation body housed within the National Institute of Standards and Technology (NIST), an agency of the United States Department of Commerce.[20] The institute was created in late 2023 as the American counterpart to the UK AI Safety Institute, with a mandate to develop measurement science for evaluating advanced artificial intelligence systems, conduct pre-deployment testing of frontier models, and coordinate with international peers through the network of national AI safety institutes.[2] By mid-2026 CAISI had completed more than 40 model evaluations, including assessments of unreleased systems, and operated pre-deployment testing agreements with five frontier developers: Anthropic, OpenAI, Google DeepMind, Microsoft, and xAI.[21][22]

Its inaugural director, Elizabeth Kelly, was appointed in February 2024 and led the institute through its first year of operation, including the August 2024 memoranda of understanding with Anthropic and OpenAI and the November 2024 inaugural convening of the International Network of AI Safety Institutes in San Francisco.[4][8][11] Kelly stepped down on 4 February 2025, two weeks after President Donald Trump returned to office, revoked Executive Order 14110, and signed Executive Order 14179 redirecting federal AI policy from risk mitigation toward innovation and global competitiveness.[13][12] On 3 June 2025 Commerce Secretary Howard Lutnick announced that the institute would be reorganized as the Center for AI Standards and Innovation, dropping the word "safety" from its title and reorienting its mission around national security risks and what the administration termed "pro-innovation" standards.[17] As of mid-2026, CAISI continues to operate within NIST under a new director, Chris Fall, and has signed pre-deployment testing agreements with Google DeepMind, Microsoft, and xAI alongside its existing arrangements with Anthropic and OpenAI; over its first two years it has completed more than 40 model assessments, including evaluations of unreleased systems and a high-profile 2025 study of the PRC developer DeepSeek.[21][22][31]

Background

The US AI Safety Institute is the product of two converging strands of policy. The first was the rapid acceleration of frontier model capabilities during 2022 and 2023, particularly with the public release of ChatGPT in November 2022 and the introduction of GPT-4 in March 2023. By mid-2023, the Biden administration had already secured a series of voluntary commitments from leading AI developers, including red-teaming pledges, watermarking research, and information sharing on model weights and security practices. These commitments were widely seen as insufficient given the pace of capability growth, and Congress had not yet passed any comprehensive AI legislation.

The second strand was international. In April 2023 the United Kingdom government announced the Frontier AI Taskforce, an internal team focused on building government capacity for the technical evaluation of frontier AI models, and signaled that it would host the world's first major intergovernmental conference on frontier AI risks at Bletchley Park in November 2023. UK officials lobbied other G7 governments to make parallel investments in evaluation capacity. The United States response was to create its own evaluation body, anchored at NIST, which had decades of experience publishing voluntary technical standards and had already produced the AI Risk Management Framework (AI RMF 1.0) in January 2023.

NIST's existing AI work made it the natural home for the new institute. The agency's National Cybersecurity Center of Excellence, its Computer Security Resource Center, and its Information Technology Laboratory had all built relationships with industry on standards-setting, and Elham Tabassi, then chief AI advisor at NIST, had led the development of the AI RMF and was already a recognized figure in the international AI policy community.[4]

When was the US AI Safety Institute established?

Vice President Kamala Harris, attending the AI Safety Summit at Bletchley Park on 1 November 2023, announced the formation of the US AI Safety Institute in a speech at the US embassy in London.[2] Harris's announcement was timed to coincide with Prime Minister Rishi Sunak's launch of the UK AI Safety Institute and the signing of the Bletchley Declaration.[3] Her speech framed the institute's role as developing technical guidance for regulators on issues including content authentication, watermarking of AI-generated outputs, algorithmic discrimination, and privacy.[2]

Harris's announcement built on Executive Order 14110, "Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence," which President Joe Biden had signed two days earlier on 30 October 2023.[1] Section 4.1(a) of EO 14110 directed the Secretary of Commerce, acting through NIST, to establish guidelines and best practices for the development and deployment of safe, secure, and trustworthy AI systems and to develop "companion resources" to the AI RMF.[1] The executive order also instructed the Secretary of Commerce to designate developers of dual-use foundation models above a 10^26 floating-point operation training threshold as subject to ongoing reporting requirements under the Defense Production Act, and to direct NIST to develop standards for red-teaming, content authentication, and security testing of these models.[1]

The institute was formally stood up over the following months. On 8 February 2024, Commerce Secretary Gina Raimondo announced its initial executive leadership and the launch of the AI Safety Institute Consortium.[4][5] Raimondo's announcement at the time described the institute as having a mandate to operate as the federal government's primary contact point for industry on AI safety testing and to coordinate with international partners.[4]

What is the US AI Safety Institute responsible for?

Until June 2025, the institute's mission as published on its NIST landing page was "to advance the science, practice, and adoption of AI safety across the spectrum of risks, including those to national security, public safety, and individual rights." Its activities clustered around three operational priorities.

The first was developing measurement and evaluation methodology. The institute drew on NIST's expertise in metrology to design empirical tests of model capabilities in domains of public concern: cybersecurity, biological and chemical misuse risk, software autonomy, and the robustness of model safeguards against jailbreaks and adversarial use. It worked closely with industry, the Frontier Model Forum, and academic groups to translate informal capability assessments into reproducible benchmarks.

The second was conducting pre-deployment evaluations of frontier models from US developers. Unlike the UK AI Safety Institute, which had access to Crown computing resources and was structured as a startup-like operational unit, the US institute relied on memoranda of understanding with private developers, NIST's existing technical staff, and contractor support. Its first formal evaluations were conducted jointly with the UK institute under a US-UK partnership agreement signed in April 2024.

The third was producing voluntary standards and guidance. NIST's existing publication apparatus, including its Special Publication and Internal Report series, gave the institute a mature pathway for releasing draft profiles, guidance documents, and risk assessments for public comment. In July 2024 NIST published the Generative AI Profile of the AI RMF (NIST AI 600-1), which became the principal voluntary reference document for industry compliance with the executive order's safety expectations.

The institute was structured as an office within NIST's headquarters operations rather than as an independent agency or laboratory. It did not have regulatory authority. Its findings could inform other agencies (including the Department of Homeland Security, the Department of Defense, the Bureau of Industry and Security, and the Federal Trade Commission), but the institute itself could not compel disclosure, mandate testing, or fine non-compliant developers. This structural limitation, repeatedly identified by outside commentators, reflected both the absence of comprehensive AI legislation and the politically contested nature of the executive order that authorized the institute.

Under CAISI, the operational mission was organized into three workstreams: a Frontier Assessment team running pre-deployment and post-deployment evaluations and producing classified and unclassified briefings; a Standards and Best Practices group publishing NIST guidance and managing the AISIC consortium; and an International Engagement function coordinating with allied evaluation bodies.[20] A 2026 USAJOBS posting for the Frontier Assessment team described responsibilities including the design of novel evaluations and the assessment of "economically important capabilities."[45]

Who leads the US AI Safety Institute?

The institute's executive team was assembled in two waves during early 2024.

Elizabeth Kelly was named inaugural director on 7 February 2024.[4] Before joining NIST she had served as a special assistant to President Biden for economic policy at the National Economic Council, and she was one of the principal drafters of Executive Order 14110.[26] Her appointment signaled that the administration intended the institute to operate at the intersection of technical evaluation and economic policy. Kelly held the role until 4 February 2025, when she announced her departure in a LinkedIn post that did not specify her next position.[13] Reuters and Bloomberg reported the departure as "leaving the institute's direction in question" given the new administration's stated opposition to the executive order that had authorized her work.[14][13]

Elham Tabassi was named the institute's first chief technology officer in the same February 2024 announcement.[4] Tabassi had led the development of the AI Risk Management Framework and was a long-tenured NIST scientist. She remained in her role through the 2025 transition and continues to serve in technical leadership at CAISI.

In April 2024, Secretary Raimondo announced an expanded leadership team that brought five new senior figures into the institute.[6]

Role	Name	Background
Director	Elizabeth Kelly	Former White House special assistant for economic policy; principal drafter of EO 14110
Chief Technology Officer	Elham Tabassi	Long-tenured NIST scientist; lead author of the AI Risk Management Framework
Head of AI Safety	Paul Christiano	Former head of language model alignment at OpenAI; founder of the Alignment Research Center
Chief Vision Officer	Adam Russell	Former director of the AI Division at USC's Information Sciences Institute; former DARPA program manager
Acting Chief Operating Officer and Chief of Staff	Mara Quintero Campbell	Former deputy chief operating officer at the Commerce Department's Economic Development Administration
Senior Advisor	Rob Reich	Stanford political scientist; co-author of "System Error"
Head of International Engagement	Mark Latonero	Former lead of human rights and AI policy work at the UN and World Economic Forum

The most controversial appointment was Christiano's. As founder of the Alignment Research Center, where he had pioneered work on reinforcement learning from human feedback and on capability evaluations of frontier models, Christiano was widely respected by AI safety researchers but was also publicly associated with effective altruism and the long-termist position that loss-of-control risks from advanced AI represented a central category of concern.[27] VentureBeat reported in March 2024 that NIST staff had circulated internal protests over the planned appointment, with some threatening to resign on grounds that Christiano's intellectual associations would compromise the institute's perceived neutrality.[7] Raimondo proceeded with the appointment in April 2024, and Christiano remained in role through the 2025 transition.[27]

After Kelly's departure in February 2025, the directorship sat vacant for over a year. Day-to-day operations were managed by Tabassi and a rotating set of acting senior officials while the Trump administration completed its review of the executive order's implementation activities. On 28 April 2026, the Commerce Department announced that Chris Fall, a former Department of Energy official from the first Trump administration with a background in scientific research administration, would lead the renamed CAISI.[21] Fall had previously served as director of the Department of Energy's Office of Science and then as director of ARPA-E, and was framed in administration messaging as a "measurement science" leader rather than an AI policy specialist.[21] His first public statement framed CAISI's mission as "independent, rigorous measurement science" focused on national security implications of frontier AI.[21]

Fall's appointment coincided with reporting that the administration had withdrawn an earlier offer to Collin Burns, a technical researcher previously associated with OpenAI's superalignment team, after objections that Burns was insufficiently aligned with the administration's pro-innovation messaging.[44] By spring 2026 Christiano's status was reported inconsistently, with most coverage continuing to refer to him as head of AI safety at CAISI while noting that his portfolio had been narrowed.

AISI Consortium

On 8 February 2024, alongside Kelly's appointment, Secretary Raimondo announced the AI Safety Institute Consortium (AISIC), described as the first US government consortium dedicated specifically to AI safety.[5] The consortium was organized under NIST's Cooperative Research and Development Agreement (CRADA) framework, which allows NIST to enter formal collaboration with industry and academic partners on standards-setting work.[5]

The consortium launched with more than 200 participating organizations and grew to over 280 members by the end of 2024, making it among the largest standards consortia in any domain.[5] Members spanned major AI developers (Anthropic, Google, Microsoft, OpenAI, Meta, Cohere, IBM, Amazon Web Services), enterprise software firms (Salesforce, Cisco, Adobe, Workday, Notion), cloud providers (Oracle, NVIDIA, Hewlett Packard Enterprise), academic and research institutions (Stanford, MIT, Carnegie Mellon, the Federation of American Scientists, the Center for Democracy and Technology), and industry associations.[5]

The consortium's working groups focused on five technical areas:

Working Group	Focus
1: Risk Management for Generative AI	Adapting the AI RMF to generative AI use cases
2: Synthetic Content	Developing standards for content authentication and watermarking
3: Capability Evaluations	Methodology for measuring model capabilities in security-relevant domains
4: Red-teaming	Standards for adversarial testing of frontier models
5: Safety and Security	Operational practices for safe AI deployment

Under Kelly, the consortium served as a vehicle for translating the institute's research priorities into industry-recognized practice. Working Groups 2 and 3 produced public draft documents during 2024, and the consortium held its first plenary meeting in early 2025 to review progress and set 2025 priorities. After the June 2025 reorganization, the consortium continued to operate, though several civil-society members have publicly noted reduced engagement on bias, equity, and transparency topics that had figured prominently in the original AI RMF.

How does the US AI Safety Institute test frontier models?

The institute's most operationally distinctive activity has been the pre-deployment evaluation of frontier models. Because NIST cannot compel access, all evaluations have proceeded under voluntary memoranda of understanding with developers.

On 29 August 2024, the institute announced its first formal MOUs with Anthropic and OpenAI.[8][28] Each agreement established a framework for the institute to receive access to major new models from the two companies prior to and following their public release, to conduct collaborative research on capability and safety evaluation methodology, and to provide feedback on potential safety improvements.[8] The MOUs also explicitly contemplated coordination with the UK AI Safety Institute, reflecting the joint testing partnership the two governments had announced in April 2024.[8]

The substantive details of the agreements were not made public, but the institute confirmed that pre-deployment access was central. OpenAI CEO Sam Altman publicly endorsed pre-release testing in an X post the same day.[28] Anthropic CEO Dario Amodei wrote in a December 2024 post that the agreement was modeled on Anthropic's responsible scaling policy and gave the institute access to research on safeguard robustness and dangerous capability evaluations.

The principal joint US-UK evaluations conducted under the Anthropic and OpenAI MOUs in 2024 are summarized below.

Date	Model	Co-evaluator	Domains tested
October 2024	Anthropic Claude 3.5 Sonnet (upgraded)	UK AISI	Cybersecurity, biology, software/AI development, safeguard robustness
November 2024	OpenAI o1 (preview)	UK AISI	Cybersecurity, biology, software/AI development
December 2024	OpenAI o1 (release version)	UK AISI	Cybersecurity, biology, software/AI development

The Claude 3.5 Sonnet evaluation, published as a joint US-UK report in November 2024, found that the upgraded Sonnet solved 32.5 percent of the 40 publicly available cybersecurity challenges in the US institute's test suite, compared with a 35 percent solve rate from the best reference model.[9] UK colleagues found a 36 percent solve rate on a different suite of 47 challenges (15 public, 32 private) at apprentice-level difficulty, exceeding the best reference model's 29 percent.[9] On safeguard robustness, the UK institute found that publicly available and privately developed jailbreaks could routinely circumvent the Sonnet 3.5 safeguards as evaluated.[9] Both institutes flagged biology benchmarks as the most uncertain area, since claims of useful uplift to malicious actors require subject-matter judgment that automated evaluations cannot provide.[9]

The joint evaluation of OpenAI's o1, released as a NIST publication in December 2024, was the first formal joint evaluation of an OpenAI o1 model and the first major collaborative output of the US-UK partnership.[10] The institutes reported that o1 performed roughly on par with reference models across cybersecurity, biology, and software development domains, with the notable exception of additional capabilities in cryptography-related cybersecurity tasks.[10] The model solved 36 percent of apprentice-level cybersecurity challenges in the UK suite, against 46 percent for the best reference model.[10] The institutes' joint write-up emphasized methodological caveats: tests were conducted under tight time constraints, with finite resources, and the domains assessed represented only a subset of conceivable concerns.[10]

Expansion to a five-lab program in 2026

On 5 May 2026 CAISI announced new pre-deployment testing agreements with Google DeepMind, Microsoft, and xAI, bringing the number of frontier developers in the program to five (alongside the carried-over Anthropic and OpenAI MOUs from 2024).[22][35] The new agreements were framed explicitly as national security testing rather than as safety evaluation.[22] Several outlets noted that the renegotiated terms allowed pre-deployment evaluation of company models in classified environments, a meaningful change from the 2024 MOUs that had limited the institutes' ability to test sensitive cyber or CBRN capabilities at full intensity.[36][39] In CAISI's announcement, the center stated that "to thoroughly evaluate national security-related capabilities and risks, developers frequently provide CAISI with models that have reduced or removed safeguards."[22]

CAISI confirmed at the time that the center had completed more than 40 evaluations in its first two years, including evaluations of unreleased state-of-the-art systems.[22] Some were published openly; others were shared only with the intelligence community, the Department of Defense, and the developers concerned.[39]

What did CAISI find when it evaluated DeepSeek?

The most prominent CAISI evaluations in 2025 and 2026 concerned DeepSeek, the PRC developer whose R1 model had attracted attention since its January 2025 release. In a report released on 30 September 2025, CAISI evaluated multiple DeepSeek models, including R1 and R1-0528, on cybersecurity, agentic robustness, censorship, data sharing, and capability benchmarks across 19 benchmarks.[31][32]

CAISI reported that DeepSeek models lagged the US frontier in raw capability but exhibited substantially weaker safeguards.[31] With publicly available jailbreak prompts, DeepSeek models produced detailed outputs for phishing, malware steps, and other restricted uses in 95 to 100 percent of test cases, compared with a 5 to 12 percent compliance rate for US frontier models on the same suite.[32][46] Agents built on DeepSeek's most secure model (R1-0528) were on average 12 times more likely than US frontier model agents to follow malicious instructions designed to derail them from user tasks.[32] The report also documented state-aligned censorship of politically sensitive topics, training data biased toward Chinese government positions, and an architectural finding that the consumer DeepSeek application shared user data with third parties, including ByteDance servers.[32][46]

In a follow-up report on 5 May 2026, CAISI published an evaluation of DeepSeek V4 Pro, the most recent open-weight release from the developer.[33] The center estimated that V4 Pro lagged the US frontier by approximately eight months on its composite capability score, an unchanged gap relative to the earlier R1 evaluation, and reported that DeepSeek had narrowed but not closed the safeguard robustness gap.[33][34]

The two DeepSeek reports were the most widely cited CAISI publications of the 2025 to 2026 period and were credited by administration officials with providing a factual basis for export-control and procurement decisions on PRC-origin AI systems.

Anthropic, Mythos, and the Pentagon dispute

The single most consequential rupture during the CAISI transition concerned Anthropic, the first signatory of an August 2024 MOU. In late 2025 and early 2026, Anthropic was in a public dispute with the Department of Defense over the application of the company's usage policies to certain classified military deployments of Claude.[42] The dispute came to a head when Anthropic refused to grant the Pentagon an exemption from its standing prohibitions on certain offensive applications.[42] In February 2026 the White House issued a directive instructing federal agencies to suspend procurement of Anthropic technology, a step covered by PBS NewsHour and Federal News Network.[42] The 2024 MOU with CAISI was not formally rescinded.

The standoff was complicated in April 2026 by Project Glasswing, an Anthropic-led cybersecurity initiative built on a powerful internal Anthropic model, Claude Mythos Preview, announced in partnership with Amazon Web Services, Apple, Broadcom, Cisco, CrowdStrike, Google, JPMorgan Chase, the Linux Foundation, Microsoft, NVIDIA, and Palo Alto Networks.[40] Anthropic committed up to $100 million in usage credits and reported that Mythos Preview had identified thousands of zero-day vulnerabilities in operating systems, browsers, and other widely deployed software during a constrained internal pilot.[40][41] The company delayed broader release of Mythos out of concern that its vulnerability-discovery capabilities created substantial uplift for offensive cyber operations.[41]

The Mythos disclosure became part of the impetus for senior administration officials to draft a proposed AI security executive order modeled in part on FDA pre-market review, under which frontier AI models would be required to undergo CAISI evaluation before public release.[38][37] As of mid-May 2026 the proposed order remained under White House study, with Axios and Federal News Network reporting that drafts had circulated through several agencies and that CAISI would be the designated reviewer.[38][39] Fortune characterized the moment as an unexpected administration "embrace" of AI oversight policies it had only recently rejected.[37] Axios separately reported that officials were drafting a plan to bring Anthropic back into federal procurement, contingent on the company accepting revised terms on military and intelligence-community use; as of mid-May 2026 no such reconciliation had been announced.[43]

How is the US AI Safety Institute funded?

The institute was repeatedly described by both supporters and critics as underfunded relative to its mandate. Initial appropriations under the Fiscal Year 2024 Consolidated Appropriations Act, enacted in March 2024, included approximately $10 million dedicated to standing up the institute. NIST secured a further $10 million through a one-time Technology Modernization Fund allocation in mid-2024. By comparison, the UK AI Safety Institute operated with annual funding of around £66 million ($83 million) and access to over £1.5 billion in public computing resources.

Fiscal year	Appropriations	Source
FY2024	~$10 million dedicated + ~$10 million one-time TMF	Consolidated Appropriations Act 2024
FY2025 (proposed)	~$50 million for AI safety activities at NIST	Biden FY2025 budget request
FY2025 (enacted)	Continuing resolution at FY2024 levels	Congressional impasse
FY2026 (enacted, January 2026)	$55 million for NIST AI work; up to $10 million specifically to expand CAISI	FY2026 appropriations
FY2026 (Trump request)	Reduced relative to Biden trajectory (specific amount not public)	Trump administration request

Biden's FY2025 budget proposal, submitted in March 2024, requested approximately $50 million for AI safety activities at NIST, including an expanded AI Safety Institute, but Congress did not enact a full FY2025 appropriations bill before the change of administration, leaving the institute on continuing resolution at FY2024 levels.[24] The January 2026 FY2026 appropriations package, negotiated during the post-CAISI transition, included approximately $55 million for NIST AI research and measurement and up to $10 million specifically directed at expanding CAISI. Federal News Network reported in May 2026 that CAISI's full-time staffing remained around 30 and that the center had received roughly $30 million in cumulative appropriations since 2024.[38] Outside policy groups argued for substantially larger budgets: the America First Policy Institute recommended $50 to $100 million in annual funding, while the Federation of American Scientists argued for up to $155 million in annual operating funds plus $155 to $275 million in one-time setup costs for high-security compute.

Senators Maria Cantwell, Todd Young, John Hickenlooper, and Marsha Blackburn's bipartisan Future of AI Innovation Act, introduced in April 2024 and reintroduced in 2026, would have codified the institute in statute and authorized increased funding, but did not become law.[23] Trade publications noted that the proposed AI security executive order under White House study would, if issued, materially expand CAISI's workload without necessarily expanding its budget.[38]

Why was the institute renamed under the Trump administration?

The political context for the institute changed sharply on 20 January 2025, when Trump revoked Executive Order 14110 within hours of taking office. Three days later, on 23 January 2025, Trump signed Executive Order 14179, "Removing Barriers to American Leadership in Artificial Intelligence," which directed senior White House officials to review all actions taken under the prior order and to identify those "inconsistent with, or presenting obstacles to" a new pro-innovation AI policy.[12] EO 14179 instructed agencies to revise or rescind such actions within 180 days.[12]

Kelly's departure on 4 February 2025 came at the start of that 180-day review period.[13] In her LinkedIn announcement she expressed continued confidence in the institute's mission.[13] Bloomberg, Reuters, and the Wall Street Journal reported the departure as the most consequential personnel change to date in federal AI policy under the new administration.[13][14] The directorship remained vacant through the spring and summer of 2025.

On 11 February 2025, Vice President JD Vance delivered a keynote address at the AI Action Summit in Paris that explicitly distanced the new administration from the Bletchley-era safety framing.[15] Vance opened his speech by saying he was "not here to talk about AI safety," but "about AI opportunity."[15] He criticized European-style precautionary regulation, signaled that the United States would oppose foreign rules constraining American AI firms, and outlined four pillars of administration AI policy: mitigating worker displacement, avoiding heavy regulation, working with allies on shared standards, and ensuring AI was free from "ideological bias."[16] The United States and the United Kingdom both declined to sign the Paris summit's joint communique, marking a public divergence from the Bletchley and Seoul process.[16]

On 3 June 2025, Commerce Secretary Howard Lutnick announced that the US AI Safety Institute would be reorganized as the Center for AI Standards and Innovation (CAISI).[17] The announcement framed CAISI as "pro-innovation" and "pro-science" and stated that the renamed center would serve as "industry's primary point of contact within the U.S. government to facilitate testing and collaborative research related to harnessing and securing the potential of commercial AI systems."[17] Lutnick said in his statement, "For far too long, censorship and regulations have been used under the guise of national security. Innovators will no longer be limited by these standards."[17]

The reorganization preserved CAISI within NIST and left the AISIC consortium in place, but it narrowed the priority focus to demonstrable national security risks (cybersecurity, biosecurity, chemical weapons-related uplift, and what Lutnick described as "malign foreign influence" from adversaries' AI systems).[17] The mission document removed earlier language on algorithmic discrimination, equity, and content provenance for AI-generated media.[18] Reporting in Technical.ly and FedScoop indicated significant staff turnover during 2025, with several senior researchers leaving for industry positions.[19][18] Christiano remained in role through the transition. Tabassi continued as chief AI advisor and CAISI's senior technical official.

On 28 April 2026, the Commerce Department confirmed Chris Fall as CAISI's director.[21] Fall, who had previously led the Department of Energy's Office of Science and ARPA-E during the first Trump administration, was framed as a "measurement science" leader rather than an AI policy specialist.[21] His confirmation was followed in early May 2026 by the announcement of new pre-deployment testing agreements between CAISI and Google DeepMind, Microsoft, and xAI, modeled on the 2024 Anthropic and OpenAI MOUs but explicitly framed as national security testing rather than safety evaluation.[22] Anthropic, which had become embroiled in a separate dispute with the Pentagon over guardrails on military uses of Claude, did not sign a CAISI-branded agreement in May 2026 and at the time of writing remained outside the renamed framework on the procurement side, although Anthropic continued to cooperate with the center on selected technical work.[42] OpenAI's status was ambiguous: the company had signaled continued cooperation with CAISI but had not publicly transitioned its 2024 MOU into a CAISI-branded agreement.

The cumulative effect of the 2025 to 2026 transition was a partial reversal of the founding policy frame, but several elements of the original construct remained intact: the consortium, the MOU-based testing model, much of the technical staff, and the public commitment to "measurement science." Observers including Axios, Fortune, and the Mossavar-Rahmani Center at Harvard's Kennedy School characterized the May 2026 announcements as a partial reconvergence between the Trump administration and the Biden-era oversight architecture under a national security framing.[39][37][30]

International cooperation

From its earliest months, the US AISI was framed as the American hub of a coordinated international response to frontier AI risk, with the UK as its principal partner. In April 2024, US and UK officials announced a bilateral partnership agreement under which the two institutes would conduct at least one joint pre-deployment evaluation per year, share research methodology, and coordinate on capability and safeguard testing. The first joint deliverables under that agreement were the November 2024 Claude 3.5 Sonnet evaluation and the December 2024 OpenAI o1 evaluation.[9][10]

At the AI Seoul Summit in May 2024, ten countries plus the European Union signed the Seoul Statement of Intent toward International Cooperation on AI Safety Science, formally launching the International Network of AI Safety Institutes.[11] The network was conceived as a permanent forum for coordinating evaluation methodology, pooling research, and sharing findings with member governments. The United States hosted the network's inaugural in-person convening in San Francisco on 20 and 21 November 2024, with delegations from Australia, Canada, the European Union, France, Japan, Kenya, the Republic of Korea, Singapore, the United Kingdom, and the United States.[11] The convening produced a joint mission statement, an $11 million pooled commitment to research on synthetic content, the network's first multilateral testing exercise, and a joint statement on risk assessments for advanced AI systems.[11]

The Seoul process was the high-water mark of network-style international cooperation. The Paris AI Action Summit in February 2025 marked a public split: Vance's speech, the US and UK refusal to sign the communique, and the renaming of the UK institute to the UK AI Security Institute the following week (announced by Technology Secretary Peter Kyle at the Munich Security Conference on 14 February 2025) all signaled that the original safety-first framing was no longer the dominant axis of cooperation.[15][16] Under CAISI, US participation in the network has continued but with explicit emphasis on national security applications and on countering what the administration describes as "authoritarian influence" in AI standards-setting.

In February 2026, NIST published a consensus document on best practices for automated evaluations developed by the international network, which had been renamed the International Network for Advanced AI Measurement, Evaluation, and Science (INAMES) by the UK institute earlier that year.[29] The renaming was less universally adopted in the United States than in the United Kingdom, and CAISI publications in 2026 generally retained the older "International Network of AI Safety Institutes" branding for cross-government communications.

How does CAISI differ from the UK AI Safety Institute?

The US institute operated under structural and political constraints that distinguished it sharply from its UK peer. The UK institute was, in the description of its own founding documents, "a startup in government," with significant operational autonomy from the broader UK civil service, its own £66 million annual budget, and access to public computing resources at a scale not available to most government bodies. The US institute, by contrast, was an office within NIST, dependent on annual congressional appropriations, with a budget roughly an order of magnitude smaller in its first year, and without independent access to compute.

Dimension	US AI Safety Institute (later CAISI)	UK AI Safety Institute (later UK AISI)
Parent body	NIST (Commerce Department)	DSIT (Cabinet Office until 2025)
Founded	November 2023 announced; February 2024 stood up	November 2023
Founding director	Elizabeth Kelly	Ian Hogarth (Chair)
First year budget	~$10 million (FY2024)	£100 million (Frontier AI Taskforce, FY23/24)
Annual budget by 2025	~$10-20 million effective	~£66 million
Annual budget by 2026	~$30 million cumulative since 2024	Comparable to 2025
Structure	Office within standards agency	Operationally autonomous, startup-style
Regulatory authority	None	None
Primary mode	Voluntary standards, MOU-based testing	MOU-based testing, in-house research
Compute access	Limited; dependent on partners	National AI Research Resource, Isambard-AI
Pre-deployment evaluations published	Joint with UK (Sonnet 3.5, o1); independent DeepSeek studies (2025-26); >40 cumulative evaluations	30+ models; many with US AISI
Renamed	June 2025 to CAISI	February 2025 to UK AI Security Institute

In substantive output, the UK institute was the more productive partner. By the end of 2025 it had published more than 30 model evaluations, the open-source Inspect framework, the ControlArena platform for AI control research, and the Frontier AI Trends Report. The US institute's published evaluation output was largely confined to the joint reports with the UK and to the AI RMF Generative AI Profile, although the September 2025 and May 2026 DeepSeek studies brought CAISI's public output closer to parity in policy salience if not in volume.[31][33] This asymmetry reflected differences in budget, political support, and institutional autonomy, not in technical talent.

Reception

Reception of the institute can be sorted into three broad reactions: enthusiastic support from the AI safety and AI alignment research communities, qualified support from industry, and critical reactions from civil society groups concerned about scope and capture.

Researchers in the alignment community welcomed the institute as a long-overdue federal investment in technical evaluation capacity. Christiano's appointment in particular was read as a signal that the institute would take seriously the loss-of-control risks that his Alignment Research Center had emphasized.[27] Yoshua Bengio, Geoffrey Hinton, and a number of the signatories of the May 2023 Statement on AI Risk publicly endorsed the institute's creation.

Industry response was generally positive but measured. The AISIC's rapid growth to over 280 members reflected real industry interest in voluntary standards as an alternative to regulation, and major developers were willing to sign MOUs giving the institute pre-deployment access.[5] At the same time, executives at Anthropic, OpenAI, Google DeepMind, and Microsoft publicly argued that the institute's mandate should be "pro-innovation" and should not become a backdoor to regulation. The Business Software Alliance and similar industry associations welcomed the June 2025 transformation to CAISI on those grounds.[17] After the May 2026 expansion to a five-lab program, leading developers welcomed the agreements as confirmation that CAISI would remain a credible technical partner, though some commentators noted that a mandatory pre-deployment review would fall most heavily on smaller frontier developers.[35]

Civil society responses were more critical. The Center for Democracy and Technology, the AI Now Institute, the Federation of American Scientists, and the World Privacy Forum were active members of the AISIC and used their participation to push for expanded attention to algorithmic discrimination, labor impacts, and democratic accountability. After the Trump administration's renaming of the institute to CAISI and the apparent dropping of bias and fairness work from the priority list, several of these groups issued statements expressing concern that the United States no longer had a federal evaluation body covering the full range of AI risks.[18]

A recurring criticism, shared across reactions, was the institute's small budget and its dependence on voluntary cooperation. Without statutory authority to compel testing, without independent compute resources, and without comprehensive AI legislation backing it, the institute's leverage was always reputational. The 2025 to 2026 DeepSeek studies and the subsequent five-lab MOU expansion changed this picture somewhat, but did not resolve the underlying funding and authority constraints.

The 2025 transition to CAISI sharpened these critiques. Some observers, including the Mossavar-Rahmani Center at Harvard's Kennedy School, characterized the renaming as primarily semantic, arguing that the underlying technical work continued largely intact.[30] Others viewed the shift as a meaningful narrowing of scope away from the broader public-interest framing of the original AI RMF. The proposed AI security executive order under White House study in May 2026, which would direct an FDA-style pre-deployment review through CAISI, has been interpreted by some commentators as evidence that the administration is incrementally restoring elements of the original safety-oriented agenda under a national security framing.[37][38]

See also

References

The White House. "Executive Order on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence." 30 October 2023. https://bidenwhitehouse.archives.gov/briefing-room/presidential-actions/2023/10/30/executive-order-on-the-safe-secure-and-trustworthy-development-and-use-of-artificial-intelligence/
Deadline. "Kamala Harris To Announce U.S. AI Safety Institute As VP Attends Global Summit On The Emerging Technology." 1 November 2023. https://deadline.com/2023/10/ai-joe-biden-kamala-harris-artificial-intelligence-1235589191/
Washington Post. "Bletchley Declaration on AI offers roadmap to limit the risks." 1 November 2023. https://www.washingtonpost.com/technology/2023/11/01/ai-kamala-harris-summit/
NIST. "U.S. Commerce Secretary Gina Raimondo Announces Key Executive Leadership at U.S. AI Safety Institute." 8 February 2024. https://www.nist.gov/news-events/news/2024/02/us-commerce-secretary-gina-raimondo-announces-key-executive-leadership-us
NIST. "Biden-Harris Administration Announces First-Ever Consortium Dedicated to AI Safety." 8 February 2024. https://www.nist.gov/news-events/news/2024/02/biden-harris-administration-announces-first-ever-consortium-dedicated-ai
NIST. "U.S. Commerce Secretary Gina Raimondo Announces Expansion of U.S. AI Safety Institute Leadership Team." 16 April 2024. https://www.nist.gov/news-events/news/2024/04/us-commerce-secretary-gina-raimondo-announces-expansion-us-ai-safety
VentureBeat. "NIST staffers revolt against expected appointment of 'effective altruist' AI researcher to US AI Safety Institute." 7 March 2024. https://venturebeat.com/ai/nist-staffers-revolt-against-potential-appointment-of-effective-altruist-ai-researcher-to-us-ai-safety-institute
NIST. "U.S. AI Safety Institute Signs Agreements Regarding AI Safety Research, Testing and Evaluation With Anthropic and OpenAI." 29 August 2024. https://www.nist.gov/news-events/news/2024/08/us-ai-safety-institute-signs-agreements-regarding-ai-safety-research
NIST. "Pre-Deployment Evaluation of Anthropic's Upgraded Claude 3.5 Sonnet." November 2024. https://www.nist.gov/news-events/news/2024/11/pre-deployment-evaluation-anthropics-upgraded-claude-35-sonnet
NIST. "Pre-Deployment Evaluation of OpenAI's o1 Model." December 2024. https://www.nist.gov/news-events/news/2024/12/pre-deployment-evaluation-openais-o1-model
NIST. "FACT SHEET: U.S. Department of Commerce & U.S. Department of State Launch the International Network of AI Safety Institutes at Inaugural Convening in San Francisco." November 2024. https://www.nist.gov/news-events/news/2024/11/fact-sheet-us-department-commerce-us-department-state-launch-international
The White House. "Removing Barriers to American Leadership in Artificial Intelligence." 23 January 2025. https://www.whitehouse.gov/presidential-actions/2025/01/removing-barriers-to-american-leadership-in-artificial-intelligence/
Bloomberg. "Head of US AI Safety Institute to Leave as Trump Shifts Course." 4 February 2025. https://www.bloomberg.com/news/articles/2025-02-04/head-of-us-ai-safety-institute-to-leave-as-trump-shifts-course
Reuters. "U.S. AI Safety Institute director leaves role." 5 February 2025. https://www.usnews.com/news/politics/articles/2025-02-05/u-s-ai-safety-institute-director-leaves-role
The American Presidency Project. "Remarks by the Vice President at the Artificial Intelligence Action Summit in Paris, France." 11 February 2025. https://www.presidency.ucsb.edu/documents/remarks-the-vice-president-the-artificial-intelligence-action-summit-paris-france
Washington Post. "Vance pushes 'America First' AI agenda, accuses allies of overregulation." 11 February 2025. https://www.washingtonpost.com/politics/2025/02/11/vance-paris-ai/
U.S. Department of Commerce. "Statement from U.S. Secretary of Commerce Howard Lutnick on Transforming the U.S. AI Safety Institute into the Pro-Innovation, Pro-Science U.S. Center for AI Standards and Innovation." 3 June 2025. https://www.commerce.gov/news/press-releases/2025/06/statement-us-secretary-commerce-howard-lutnick-transforming-us-ai
FedScoop. "Trump administration rebrands AI Safety Institute." June 2025. https://fedscoop.com/trump-administration-rebrands-ai-safety-institute-aisi-caisi/
Technical.ly. "US Commerce Department renames, refocuses AI Safety Institute." 2025. https://technical.ly/civics/ai-safety-institute-overhaul-howard-lutnick/
NIST. "Center for AI Standards and Innovation (CAISI)." https://www.nist.gov/caisi
MeriTalk. "Trump Administration Taps Chris Fall to Lead CAISI." April 2026. https://www.meritalk.com/articles/trump-administration-taps-chris-fall-to-lead-caisi/
NIST. "CAISI Signs Agreements Regarding Frontier AI National Security Testing With Google DeepMind, Microsoft and xAI." May 2026. https://www.nist.gov/news-events/news/2026/05/caisi-signs-agreements-regarding-frontier-ai-national-security-testing
U.S. Senate Committee on Commerce, Science, & Transportation. "Commerce Committee Passes Bipartisan Bill to Ensure U.S. Leads Global AI Innovation." 2024. https://www.commerce.senate.gov/2024/3/cantwell-colleagues-lead-effort-to-establish-u-s-artificial-intelligence-safety-institute
Mintz. "AI Provisions in Biden's FY 2025 Budget Proposal." March 2024. https://www.mintz.com/insights-center/viewpoints/54731/2024-03-28-ai-provisions-bidens-fy-2025-budget-proposal-ai
Wikipedia. "Artificial intelligence safety institute." https://en.wikipedia.org/wiki/AI_Safety_Institute
CSIS. "The U.S. Vision for AI Safety: A Conversation with Elizabeth Kelly, Director of the U.S. AI Safety Institute." https://www.csis.org/events/us-vision-ai-safety-conversation-elizabeth-kelly-director-us-ai-safety-institute
CDO Magazine. "Ex-OpenAI Researcher Paul Christiano Named AISI's Head of AI Safety." April 2024. https://www.cdomagazine.tech/leadership-moves/ex-openai-researcher-paul-christiano-named-aisis-head-of-ai-safety
CNBC. "OpenAI and Anthropic agree to let U.S. AI Safety Institute test and evaluate new models." 29 August 2024. https://www.cnbc.com/2024/08/29/openai-and-anthropic-agree-to-let-us-ai-safety-institute-test-models.html
NIST. "International Network for Advanced AI Measurement, Evaluation, and Science Publishes Consensus Areas on Practices for Automated Evaluations." February 2026. https://www.nist.gov/news-events/news/2026/02/international-network-advanced-ai-measurement-evaluation-and-science
Harvard Kennedy School Mossavar-Rahmani Center. "From Safety to Secure Innovation" and "Renaming the US AI Safety Institute Is About Priorities, Not Semantics." 2025. https://www.hks.harvard.edu/centers/mrcbg/programs/growthpolicy/safety-secure-innovation
NIST. "CAISI Evaluation of DeepSeek AI Models Finds Shortcomings and Risks." 30 September 2025. https://www.nist.gov/news-events/news/2025/09/caisi-evaluation-deepseek-ai-models-finds-shortcomings-and-risks
NIST. "Evaluation of DeepSeek AI Models" (PDF). 30 September 2025. https://www.nist.gov/system/files/documents/2025/09/30/CAISI_Evaluation_of_DeepSeek_AI_Models.pdf
NIST. "CAISI Evaluation of DeepSeek V4 Pro." 5 May 2026. https://www.nist.gov/news-events/news/2026/05/caisi-evaluation-deepseek-v4-pro
Digital Watch Observatory. "DeepSeek V4 trails US frontier by eight months, according to CAISI evaluation." May 2026. https://dig.watch/updates/deepseek-v4-pro-caisi-us-nist-evaluation
Nextgov/FCW. "Commerce AI center will evaluate Google Deepmind, Microsoft and xAI models." 5 May 2026. https://www.nextgov.com/artificial-intelligence/2026/05/commerce-ai-center-will-evaluate-google-deepmind-microsoft-and-xai-models/413349/
CNN Business. "Microsoft, Google and xAI will let the government test their AI models before launch." 5 May 2026. https://www.cnn.com/2026/05/05/tech/microsoft-google-xai-government-test-ai-models
Fortune. "Trump administration suddenly embraces AI oversight ideas it once rejected." 6 May 2026. https://fortune.com/2026/05/06/trump-administration-embraces-ai-oversight-policies-it-once-rejected-anthropic-mythos-caisi/
Federal News Network. "WH 'studying' AI security executive order." May 2026. https://federalnewsnetwork.com/artificial-intelligence/2026/05/wh-studying-ai-security-executive-order/
Axios. "U.S. ramps up frontier AI testing as White House pivots toward safety." 5 May 2026. https://www.axios.com/2026/05/05/us-frontier-ai-testing-white-house-pivots-safety
Anthropic. "Project Glasswing: Securing critical software for the AI era." April 2026. https://www.anthropic.com/glasswing
NPR. "How AI is getting better at finding security holes." 11 April 2026. https://www.npr.org/2026/04/11/nx-s1-5778508/anthropic-project-glasswing-ai-cybersecurity-mythos-preview
PBS NewsHour. "Trump orders federal agencies to stop using Anthropic tech over AI safety dispute." February 2026. https://www.pbs.org/newshour/politics/trump-orders-federal-agencies-to-stop-using-anthropic-tech-over-ai-safety-dispute
Axios. "Trump officials draft plan to bring Anthropic back amid Pentagon fight." 29 April 2026. https://www.axios.com/2026/04/29/trump-anthropic-pentagon-ai-executive-order-gov
The Daily Signal. "Commerce Department Changes Direction on AI Safety Leader." 23 April 2026. https://www.dailysignal.com/2026/04/23/trump-anthropic-researcher/
USAJOBS. "Member of Technical Staff, Frontier Assessment" (CAISI). 2026. https://ai.usajobs.gov/job/856265200
TechRepublic. "DeepSeek AI Models Are Unsafe and Unreliable, Finds NIST-Backed Study." October 2025. https://www.techrepublic.com/article/news-deepseek-security-gaps-caisi-study/

Editing: US AI Safety Institute (Center for AI Standards and Innovation)