US AI Safety Institute (Center for AI Standards and Innovation)

The US AI Safety Institute (USAISI or US AISI), since June 2025 the Center for AI Standards and Innovation (CAISI), and since October 2026 styled on NIST's website as the Center for Advancing Innovation and Standards for Super Intelligence (CAISSI),^[48] is a research and evaluation body housed within the National Institute of Standards and Technology (NIST), an agency of the United States Department of Commerce.^[20] The institute was created in late 2023 as the American counterpart to the UK AI Safety Institute, with a mandate to develop measurement science for evaluating advanced artificial intelligence systems, conduct pre-deployment testing of frontier models, and coordinate with international peers through the network of national AI safety institutes.^[2] By mid-2026 CAISI had completed more than 40 model evaluations, including assessments of unreleased systems, and operated pre-deployment testing agreements with five frontier developers: Anthropic, OpenAI, Google DeepMind, Microsoft, and xAI.^[21][22]

Its inaugural director, Elizabeth Kelly, was appointed in February 2024 and led the institute through its first year of operation, including the August 2024 memoranda of understanding with Anthropic and OpenAI and the November 2024 inaugural convening of the International Network of AI Safety Institutes in San Francisco.^[4][8][11] Kelly left on 5 February 2025, about two weeks after President Donald Trump returned to office, revoked Executive Order 14110, and signed Executive Order 14179 redirecting federal AI policy from risk mitigation toward innovation and global competitiveness.^[13][12] On 3 June 2025 Commerce Secretary Howard Lutnick announced that the institute would be reorganized as the Center for AI Standards and Innovation, dropping the word "safety" from its title and reorienting its mission around national security risks and what the administration termed "pro-innovation" standards.^[17] CAISI continues to operate within NIST; Chris Fall, named director in April 2026, resigned that July and NIST Director Arvind Raman became acting director.^[49] CAISI has signed pre-deployment testing agreements with Google DeepMind, Microsoft, and xAI alongside its existing arrangements with Anthropic and OpenAI; over its first two years it has completed more than 40 model assessments, including evaluations of unreleased systems and a high-profile 2025 study of the PRC developer DeepSeek.^[21][22][31]

Background

The US AI Safety Institute is the product of two converging strands of policy. The first was the rapid acceleration of frontier model capabilities during 2022 and 2023, particularly with the public release of ChatGPT in November 2022 and the introduction of GPT-4 in March 2023. By mid-2023, the Biden administration had already secured a series of voluntary commitments from leading AI developers, including red-teaming pledges, watermarking research, and information sharing on model weights and security practices. These commitments were widely seen as insufficient given the pace of capability growth, and Congress had not yet passed any comprehensive AI legislation.

The second strand was international. In April 2023 the United Kingdom government announced the Frontier AI Taskforce, an internal team focused on building government capacity for the technical evaluation of frontier AI models, and signaled that it would host the world's first major intergovernmental conference on frontier AI risks at Bletchley Park in November 2023. UK officials lobbied other G7 governments to make parallel investments in evaluation capacity. The United States response was to create its own evaluation body, anchored at NIST, which had decades of experience publishing voluntary technical standards and had already produced the AI Risk Management Framework (AI RMF 1.0) in January 2023.

NIST's existing AI work made it the natural home for the new institute. The agency's National Cybersecurity Center of Excellence, its Computer Security Resource Center, and its Information Technology Laboratory had all built relationships with industry on standards-setting, and Elham Tabassi, then chief AI advisor at NIST, had led the development of the AI RMF and was already a recognized figure in the international AI policy community.^[4]

When was the US AI Safety Institute established?

Vice President Kamala Harris, attending the AI Safety Summit at Bletchley Park on 1 November 2023, announced the formation of the US AI Safety Institute in a speech at the US embassy in London.^[2] Harris's announcement was timed to coincide with Prime Minister Rishi Sunak's launch of the UK AI Safety Institute and the signing of the Bletchley Declaration.^[3] Her speech framed the institute's role as developing technical guidance for regulators on issues including content authentication, watermarking of AI-generated outputs, algorithmic discrimination, and privacy.^[2]

Harris's announcement built on Executive Order 14110, "Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence," which President Joe Biden had signed two days earlier on 30 October 2023.^[1] Section 4.1(a) of EO 14110 directed the Secretary of Commerce, acting through NIST, to establish guidelines and best practices for the development and deployment of safe, secure, and trustworthy AI systems and to develop "companion resources" to the AI RMF.^[1] The executive order also instructed the Secretary of Commerce to designate developers of dual-use foundation models above a 10^26 floating-point operation training threshold as subject to ongoing reporting requirements under the Defense Production Act, and to direct NIST to develop standards for red-teaming, content authentication, and security testing of these models.^[1]

The institute was formally stood up over the following months. On 7 and 8 February 2024, Commerce Secretary Gina Raimondo announced its initial executive leadership and the launch of the AI Safety Institute Consortium.^[4][5] Raimondo's announcement at the time described the institute as having a mandate to operate as the federal government's primary contact point for industry on AI safety testing and to coordinate with international partners.^[4]

What is the US AI Safety Institute responsible for?

Until June 2025, the institute's mission as published on its NIST landing page was "to advance the science, practice, and adoption of AI safety across the spectrum of risks, including those to national security, public safety, and individual rights." Its activities clustered around three operational priorities.

The first was developing measurement and evaluation methodology. The institute drew on NIST's expertise in metrology to design empirical tests of model capabilities in domains of public concern: cybersecurity, biological and chemical misuse risk, software autonomy, and the robustness of model safeguards against jailbreaks and adversarial use. It worked closely with industry, the Frontier Model Forum, and academic groups to translate informal capability assessments into reproducible benchmarks.

The second was conducting pre-deployment evaluations of frontier models from US developers. Unlike the UK AI Safety Institute, which had access to Crown computing resources and was structured as a startup-like operational unit, the US institute relied on memoranda of understanding with private developers, NIST's existing technical staff, and contractor support. Its first formal evaluations were conducted jointly with the UK institute under a US-UK partnership agreement signed in April 2024.

The third was producing voluntary standards and guidance. NIST's existing publication apparatus, including its Special Publication and Internal Report series, gave the institute a mature pathway for releasing draft profiles, guidance documents, and risk assessments for public comment. In July 2024 NIST published the Generative AI Profile of the AI RMF (NIST AI 600-1), which became the principal voluntary reference document for industry compliance with the executive order's safety expectations.

The institute was structured as an office within NIST's headquarters operations rather than as an independent agency or laboratory. It did not have regulatory authority. Its findings could inform other agencies (including the Department of Homeland Security, the Department of Defense, the Bureau of Industry and Security, and the Federal Trade Commission), but the institute itself could not compel disclosure, mandate testing, or fine non-compliant developers. This structural limitation, repeatedly identified by outside commentators, reflected both the absence of comprehensive AI legislation and the politically contested nature of the executive order that authorized the institute.

Under CAISI, NIST listed the center's tasks as developing security guidelines and helping industry develop voluntary standards; establishing voluntary agreements with developers and leading unclassified evaluations of national security risks; assessing US and adversary AI systems and foreign influence; coordinating with other agencies; and representing US interests in international standards.^[20][17] A 2026 USAJOBS posting for the Frontier Assessment team described responsibilities including the design of novel evaluations and the assessment of "economically important capabilities."^[45]

Who leads the US AI Safety Institute?

The institute's executive team was assembled in two waves during early 2024.

Elizabeth Kelly was named inaugural director on 7 February 2024.^[4] Before joining NIST she had served as a special assistant to President Biden for economic policy at the National Economic Council, and CSIS described her as "a driving force behind" Executive Order 14110.^[26] Her appointment signaled that the administration intended the institute to operate at the intersection of technical evaluation and economic policy. Kelly held the role until 5 February 2025, announcing her departure in a LinkedIn post that did not specify her next position.^[14] Reuters reported that the move left open the nascent body's direction under President Trump, who had already revoked the executive order behind it.^[14]

Elham Tabassi was named the institute's first chief technology officer in the same February 2024 announcement.^[4] Tabassi had led the development of the AI Risk Management Framework and was a long-tenured NIST scientist. She left NIST in early 2025 to direct the Brookings Institution's Artificial Intelligence and Emerging Technology Initiative, starting there on 10 March 2025.^[50]

In April 2024, Secretary Raimondo announced an expanded leadership team that brought five new senior figures into the institute.^[6]

The most controversial appointment was Christiano's. A former OpenAI researcher who had pioneered work on reinforcement learning from human feedback, and founder of the Alignment Research Center, which had worked on capability evaluations of frontier models, Christiano was widely respected by AI safety researchers but was also publicly associated with effective altruism and the long-termist position that loss-of-control risks from advanced AI represented a central category of concern.^[27] VentureBeat reported in March 2024 that NIST staff had circulated internal protests over the planned appointment, with some threatening to resign on grounds that Christiano's intellectual associations would compromise the institute's perceived neutrality.^[7] Raimondo proceeded with the appointment in April 2024.^[6][27] Christiano stayed on through the 2025 transition; when he joined the OpenAI Foundation board in September 2026, OpenAI described him as a senior tech advisor at CAISI.^[51]

After Kelly's departure in February 2025, the directorship sat vacant for more than a year. In April 2026 Collin Burns, a former OpenAI and Anthropic researcher, briefly took the post but was pushed out within days after the White House learned of his Anthropic employment.^[21][39][44] Later that month NIST confirmed that Chris Fall, a former Department of Energy official from the first Trump administration, would lead CAISI.^[21] Fall had been director of the Department of Energy's Office of Science and, most recently, a vice president at MITRE.^[21] Announcing new testing agreements in May 2026, he said that "independent, rigorous measurement science is essential to understanding frontier AI and its national security implications."^[22] Fall resigned in July 2026 after about three months, and the Commerce Department named NIST Director Arvind Raman acting CAISI director.^[49]

AISI Consortium

On 8 February 2024, the day after Kelly's appointment, Secretary Raimondo announced the AI Safety Institute Consortium (AISIC), described as the first US government consortium dedicated specifically to AI safety.^[5] The consortium was organized under NIST's Cooperative Research and Development Agreement (CRADA) framework, which allows NIST to enter formal collaboration with industry and academic partners on standards-setting work.^[5]

The consortium launched with more than 200 participating organizations and grew to over 280 members by the end of 2024, making it among the largest standards consortia in any domain.^[5] Members spanned major AI developers (Anthropic, Google, Microsoft, OpenAI, Meta, Cohere, IBM, Amazon Web Services), enterprise software firms (Salesforce, Cisco, Adobe, Workday, Notion), cloud providers (Oracle, NVIDIA, Hewlett Packard Enterprise), academic and research institutions (Stanford, MIT, Carnegie Mellon, the Federation of American Scientists, the Center for Democracy and Technology), and industry associations.^[5]

The consortium's working groups focused on five technical areas:

Under Kelly, the consortium served as a vehicle for translating the institute's research priorities into industry-recognized practice. Working Groups 2 and 3 produced public draft documents during 2024, and the consortium held its first plenary meeting in early 2025 to review progress and set 2025 priorities. After the June 2025 reorganization, the consortium continued to operate (in May 2026 NIST renamed it the NIST AI Consortium and refocused it on AI measurement, innovation and adoption^[52]), though several civil-society members have publicly noted reduced engagement on bias, equity, and transparency topics that had figured prominently in the original AI RMF.

How does the US AI Safety Institute test frontier models?

The institute's most operationally distinctive activity has been the pre-deployment evaluation of frontier models. Because NIST cannot compel access, all evaluations have proceeded under voluntary memoranda of understanding with developers.

On 29 August 2024, the institute announced its first formal MOUs with Anthropic and OpenAI.^[8][28] Each agreement established a framework for the institute to receive access to major new models from the two companies prior to and following their public release, to conduct collaborative research on capability and safety evaluation methodology, and to provide feedback on potential safety improvements.^[8] The MOUs also explicitly contemplated coordination with the UK AI Safety Institute, reflecting the joint testing partnership the two governments had announced in April 2024.^[8]

The substantive details of the agreements were not made public, but the institute confirmed that pre-deployment access was central. OpenAI CEO Sam Altman publicly endorsed pre-release testing in an X post the same day.^[28]

The principal joint US-UK evaluations conducted under the Anthropic and OpenAI MOUs in 2024 are summarized below.

The Claude 3.5 Sonnet evaluation, published as a joint US-UK report in November 2024, found that the upgraded Sonnet solved 32.5 percent of the 40 publicly available cybersecurity challenges in the US institute's test suite, compared with a 35 percent solve rate from the best reference model.^[9] UK colleagues found a 36 percent solve rate on a different suite of 47 challenges (15 public, 32 private) at apprentice-level difficulty, exceeding the best reference model's 29 percent.^[9] On safeguard robustness, the UK institute found that publicly available and privately developed jailbreaks could routinely circumvent the Sonnet 3.5 safeguards as evaluated.^[9] Biology results came from the US institute alone, since the UK institute did not publish in that domain: on multiple-choice research questions the upgraded Sonnet performed comparably to reference models and well below human expert baselines, though a tool-augmented setup at times exceeded expert baselines.^[9]

The joint evaluation of OpenAI's o1, released as a NIST publication in December 2024, was the second published joint US-UK pre-deployment evaluation, after the Claude 3.5 Sonnet report, and covered OpenAI o1.^[10] The institutes reported that o1 performed roughly on par with reference models across cybersecurity, biology, and software development domains, with the notable exception of additional capabilities in cryptography-related cybersecurity tasks.^[10] The model solved 36 percent of apprentice-level cybersecurity challenges in the UK suite, against 46 percent for the best reference model.^[10] The institutes' joint write-up emphasized methodological caveats: tests were conducted under tight time constraints, with finite resources, and the domains assessed represented only a subset of conceivable concerns.^[10]

Expansion to a five-lab program in 2026

On 5 May 2026 CAISI announced new pre-deployment testing agreements with Google DeepMind, Microsoft, and xAI, bringing the number of frontier developers in the program to five (alongside the 2024 Anthropic and OpenAI agreements, which Commerce said had been renegotiated).^[22][35][39] The new agreements were framed explicitly as national security testing rather than as safety evaluation.^[22] The agreements also support testing in classified environments, with evaluators from across government taking part through the CAISI-convened TRAINS Taskforce.^[22][35] In CAISI's announcement, the center stated that "to thoroughly evaluate national security-related capabilities and risks, developers frequently provide CAISI with models that have reduced or removed safeguards."^[22]

CAISI confirmed at the time that the center had completed more than 40 evaluations in its first two years, including evaluations of unreleased state-of-the-art systems.^[22]

What did CAISI find when it evaluated DeepSeek and GLM-5.3?

The most prominent CAISI evaluations in 2025 and 2026 concerned DeepSeek, the PRC developer whose R1 model had attracted attention since its January 2025 release. In a report released on 30 September 2025, CAISI evaluated three DeepSeek models (R1, R1-0528 and V3.1) against four US models across 19 benchmarks covering performance, cost, security, CCP narratives and adoption.^[31][32]

CAISI reported that DeepSeek models lagged the US frontier in raw capability but exhibited substantially weaker safeguards.^[31] With a public jailbreak, DeepSeek V3.1 complied with 95 to 100 percent of malicious requests on harmful biology, violence, hacking and scamming (giving highly detailed answers to 62 to 70 percent), compared with 5 to 12 percent compliance for US frontier models.^[32] Agents built on DeepSeek's most secure model (R1-0528) were on average 12 times more likely than US frontier model agents to follow malicious instructions designed to derail them from user tasks.^[32] The report also found that DeepSeek models echoed four times as many inaccurate and misleading Chinese Communist Party narratives as US reference models, and that downloads of DeepSeek models had risen nearly 1,000 percent since January 2025.^[31]

In a follow-up report dated 1 May 2026, CAISI published an evaluation of DeepSeek V4 Pro, then the developer's most recent open-weight release.^[33] The center estimated that V4 Pro lagged the US frontier by approximately eight months on its composite capability score, called it the most capable PRC model it had evaluated, and found it more cost-efficient than GPT-5.4 mini on five of seven benchmarks.^[33][34]

The two DeepSeek reports were the most widely cited CAISI publications of the 2025 to 2026 period.

CAISI's assessments of PRC-developed models also covered Z.ai (formerly Zhipu AI). On 17 September 2026 it published an assessment of the cyber capabilities of GLM-5.3, an open-weight model Z.ai had released in August, having previously assessed the company's GLM-5.2. Its key findings were that "GLM-5.3 is the most cyber-capable open-weight model released to date" and that the model's cyber capabilities "are significantly lower than those of current U.S. frontier models," with GLM-5.3 lagging the U.S. frontier "by about four months in an aggregate measure of performance across CAISI cyber benchmarks."^[47] In July 2026 CAISI and the UK AI Security Institute also published a joint preliminary assessment of the cyber capabilities of Moonshot AI's Kimi K3.^[20]

Anthropic, Mythos, and the Pentagon dispute

The single most consequential rupture during the CAISI transition concerned Anthropic, the first signatory of an August 2024 MOU. In late 2025 and early 2026, Anthropic was in a public dispute with the Department of Defense over the application of the company's usage policies to certain classified military deployments of Claude.^[42] The dispute came to a head when Anthropic refused to drop its conditions that Claude not be used for mass surveillance of Americans or in fully autonomous weapons.^[42] On 27 February 2026 President Trump ordered federal agencies to stop using Anthropic technology, and Defense Secretary Pete Hegseth designated the company a supply chain risk.^[42] The 2024 MOU with CAISI was not rescinded; in May 2026 a Commerce spokesperson said the Anthropic and OpenAI partnerships were "ongoing and reflect updated MOUs."^[39]

The standoff was complicated in April 2026 by Project Glasswing, an Anthropic-led cybersecurity initiative built on a powerful internal Anthropic model, Claude Mythos Preview, announced in partnership with Amazon Web Services, Apple, Broadcom, Cisco, CrowdStrike, Google, JPMorgan Chase, the Linux Foundation, Microsoft, NVIDIA, and Palo Alto Networks.^[40] Anthropic committed up to $100 million in usage credits and reported that Mythos Preview had identified thousands of zero-day vulnerabilities in operating systems, browsers, and other widely deployed software during a constrained internal pilot.^[40][41] The company delayed broader release of Mythos out of concern that its vulnerability-discovery capabilities created substantial uplift for offensive cyber operations.^[41]

The Mythos disclosure became part of the impetus for senior administration officials to draft a proposed AI security executive order modeled in part on FDA pre-market review, under which frontier AI models would undergo government evaluation before public release.^[38][37] Federal News Network reported that such an order would likely increase CAISI's responsibilities.^[38] The order President Trump signed on 2 June 2026, "Promoting Advanced Artificial Intelligence Innovation and Security," was scaled back: it set up a voluntary framework, designed by the Treasury, the NSA and CISA in consultation with NIST, under which developers may give the government access to "covered frontier models" for up to 30 days before release, and it states that it creates no mandatory preclearance requirement.^[53] Fortune characterized the moment as an unexpected administration "embrace" of AI oversight policies it had only recently rejected.^[37] Axios separately reported that officials were drafting a plan to bring Anthropic back into federal procurement, contingent on the company accepting revised terms on military and intelligence-community use; as of mid-May 2026 no such reconciliation had been announced.^[43]

How is the US AI Safety Institute funded?

The institute was repeatedly described by both supporters and critics as underfunded relative to its mandate. Initial appropriations under the Fiscal Year 2024 Consolidated Appropriations Act, enacted in March 2024, included approximately $10 million dedicated to standing up the institute. NIST secured a further $10 million through a one-time Technology Modernization Fund allocation in mid-2024. By comparison, the UK AI Safety Institute operated with annual funding of around £66 million ($83 million) and access to over £1.5 billion in public computing resources.

Biden's FY2025 budget proposal, submitted in March 2024, requested $65 million for the Department of Commerce "to safeguard, regulate, and promote AI," including funding to establish the AI Safety Institute, but Congress did not enact a full FY2025 appropriations bill before the change of administration, leaving the institute on continuing resolution at FY2024 levels.^[24] The January 2026 FY2026 appropriations package, negotiated during the post-CAISI transition, included approximately $55 million for NIST AI research and measurement and up to $10 million specifically directed at expanding CAISI. An America First Policy Institute brief, reported by Federal News Network in May 2026, put CAISI's staff at about 30 and said the center had received about $30 million since 2024.^[38] Outside policy groups argued for substantially larger budgets: the America First Policy Institute recommended $50 to $100 million in annual funding, while the Federation of American Scientists argued for up to $155 million in annual operating funds plus $155 to $275 million in one-time setup costs for high-security compute.

Senators Maria Cantwell, Todd Young, John Hickenlooper, and Marsha Blackburn's bipartisan Future of AI Innovation Act, introduced in April 2024 and reintroduced in 2026, would have codified the institute in statute and authorized increased funding, but did not become law.^[23] Federal News Network noted in May 2026 that the then-proposed AI security executive order would likely increase CAISI's workload; the order signed in June 2026 was voluntary and assigned the main roles to the Treasury, the NSA and CISA.^[38][53]

Why was the institute renamed under the Trump administration?

The political context for the institute changed sharply on 20 January 2025, when Trump revoked Executive Order 14110 within hours of taking office. Three days later, on 23 January 2025, Trump signed Executive Order 14179, "Removing Barriers to American Leadership in Artificial Intelligence," which directed senior White House officials to review all actions taken under the prior order and to identify those "inconsistent with, or presenting obstacles to" a new pro-innovation AI policy.^[12] EO 14179 directed agencies to suspend, revise or rescind such actions as appropriate, and separately required an AI Action Plan within 180 days.^[12]

Kelly left on 5 February 2025, while that review was under way.^[14] In her LinkedIn announcement she wrote: "I am confident that AISI's future is bright and its mission remains vital to the future of AI innovation."^[14] The directorship remained vacant through the spring and summer of 2025.

On 11 February 2025, Vice President JD Vance delivered a keynote address at the AI Action Summit in Paris that explicitly distanced the new administration from the Bletchley-era safety framing.^[15] Vance opened his speech by saying he was "not here to talk about AI safety," but "about AI opportunity."^[15] He criticized European-style precautionary regulation, signaled that the United States would oppose foreign rules constraining American AI firms, and outlined four points of administration AI policy: keeping American AI the worldwide "gold standard" and partner of choice, avoiding excessive regulation, keeping AI "free from ideological bias," and maintaining a "pro-worker growth path" for AI.^[15] The United States and the United Kingdom both declined to sign the Paris summit's joint communique, marking a public divergence from the Bletchley and Seoul process.^[16]

On 3 June 2025, Commerce Secretary Howard Lutnick announced that the US AI Safety Institute would be reorganized as the Center for AI Standards and Innovation (CAISI).^[17] The announcement framed CAISI as "pro-innovation" and "pro-science" and stated that the renamed center would serve as "industry's primary point of contact within the U.S. government to facilitate testing and collaborative research related to harnessing and securing the potential of commercial AI systems."^[17] Lutnick said in his statement, "For far too long, censorship and regulations have been used under the guise of national security. Innovators will no longer be limited by these standards."^[17]

The reorganization preserved CAISI within NIST and left the AISIC consortium in place, but it narrowed the priority focus to demonstrable national security risks (cybersecurity, biosecurity, chemical weapons-related uplift, and what Lutnick described as "malign foreign influence" from adversaries' AI systems).^[17] The new mission statement made no mention of algorithmic discrimination, equity, or content provenance for AI-generated media, which had featured in the institute's Biden-era remit.^[17] Several founding leaders had already left, including Kelly and chief technology officer Elham Tabassi, who joined the Brookings Institution in March 2025.^[50] Christiano stayed on.^[51]

In late April 2026 NIST confirmed Chris Fall as CAISI's director; he had led the Department of Energy's Office of Science during the first Trump administration, and he resigned in July 2026.^[21][49] His confirmation was followed in early May 2026 by the announcement of new pre-deployment testing agreements between CAISI and Google DeepMind, Microsoft, and xAI, modeled on the 2024 Anthropic and OpenAI MOUs but explicitly framed as national security testing rather than safety evaluation.^[22] A Commerce spokesperson said the existing Anthropic and OpenAI partnerships were "ongoing and reflect updated MOUs" renegotiated to the center's new directives, even as Anthropic remained in a separate dispute with the Pentagon over guardrails on military uses of Claude.^[39][42]

The cumulative effect of the 2025 to 2026 transition was a partial reversal of the founding policy frame, but several elements of the original construct remained intact: the consortium, the MOU-based testing model, much of the technical staff, and the public commitment to "measurement science." Axios and Fortune characterized the May 2026 announcements as a turn back toward Biden-era oversight ideas under a national security framing.^[39][37]

International cooperation

From its earliest months, the US AISI was framed as the American hub of a coordinated international response to frontier AI risk, with the UK as its principal partner. In April 2024, US and UK officials announced a bilateral partnership agreement under which the two institutes would conduct at least one joint pre-deployment evaluation per year, share research methodology, and coordinate on capability and safeguard testing. The first joint deliverables under that agreement were the November 2024 Claude 3.5 Sonnet evaluation and the December 2024 OpenAI o1 evaluation.^[9][10]

At the AI Seoul Summit in May 2024, ten countries plus the European Union signed the Seoul Statement of Intent toward International Cooperation on AI Safety Science, formally launching the International Network of AI Safety Institutes.^[11] The network was conceived as a permanent forum for coordinating evaluation methodology, pooling research, and sharing findings with member governments. The United States hosted the network's inaugural in-person convening in San Francisco on 20 and 21 November 2024, with delegations from Australia, Canada, the European Union, France, Japan, Kenya, the Republic of Korea, Singapore, the United Kingdom, and the United States.^[11] The convening produced a joint mission statement, more than $11 million in research funding commitments on synthetic content, the network's first multilateral testing exercise, and a joint statement on risk assessments for advanced AI systems.^[11]

The Seoul process was the high-water mark of network-style international cooperation. The Paris AI Action Summit in February 2025 marked a public split: Vance's speech, the US and UK refusal to sign the communique, and the renaming of the UK institute to the UK AI Security Institute the following week (announced by Technology Secretary Peter Kyle at the Munich Security Conference on 14 February 2025) all signaled that the original safety-first framing was no longer the dominant axis of cooperation.^[15][16] Under CAISI, US participation in the network has continued but with explicit emphasis on national security applications and on countering what the administration describes as "authoritarian influence" in AI standards-setting.

In February 2026, NIST published a consensus document on best practices for automated evaluations developed by the international network, which by then operated as the International Network for Advanced AI Measurement, Evaluation, and Science; NIST's announcement described the network as founded by CAISI in November 2024 and still made up of the same ten members.^[29]

How does CAISI differ from the UK AI Safety Institute?

The US institute operated under structural and political constraints that distinguished it sharply from its UK peer. The UK institute was, in the description of its own founding documents, "a startup in government," with significant operational autonomy from the broader UK civil service, its own £66 million annual budget, and access to public computing resources at a scale not available to most government bodies. The US institute, by contrast, was an office within NIST, dependent on annual congressional appropriations, with a budget roughly an order of magnitude smaller in its first year, and without independent access to compute.

In substantive output, the UK institute was the more productive partner. By the end of 2025 it had published more than 30 model evaluations, the open-source Inspect framework, the ControlArena platform for AI control research, and the Frontier AI Trends Report. The US institute's published evaluation output was largely confined to the joint reports with the UK and to the AI RMF Generative AI Profile, although the DeepSeek studies of September 2025 and May 2026, and the 2026 assessments of Z.ai's GLM-5.2 and GLM-5.3 and (jointly with the UK) Moonshot AI's Kimi K3, brought CAISI's public output closer to parity in policy salience if not in volume.^{[31][33][47][20]} This asymmetry reflected differences in budget, political support, and institutional autonomy, not in technical talent.

Reception

Reception of the institute can be sorted into three broad reactions: enthusiastic support from the AI safety and AI alignment research communities, qualified support from industry, and critical reactions from civil society groups concerned about scope and capture.

Researchers in the alignment community welcomed the institute as a long-overdue federal investment in technical evaluation capacity. Christiano's appointment in particular was read as a signal that the institute would take seriously the loss-of-control risks that his Alignment Research Center had emphasized.^[27] Yoshua Bengio, Geoffrey Hinton, and a number of the signatories of the May 2023 Statement on AI Risk publicly endorsed the institute's creation.

Industry response was generally positive but measured. The AISIC's rapid growth to over 280 members reflected real industry interest in voluntary standards as an alternative to regulation, and major developers were willing to sign MOUs giving the institute pre-deployment access.^[5] At the same time, executives at Anthropic, OpenAI, Google DeepMind, and Microsoft publicly argued that the institute's mandate should be "pro-innovation" and should not become a backdoor to regulation. After the May 2026 expansion to a five-lab program, the Business Software Alliance said the announcement "reinforces CAISI's role as the right institutional home within government for advancing evaluation and measurement science,"^[35] and Microsoft's chief responsible AI officer, Natasha Crampton, said CAISI offered additional "technical, scientific and national security expertise."^[36]

Civil society responses were more critical. The Center for Democracy and Technology, the AI Now Institute, the Federation of American Scientists, and the World Privacy Forum were active members of the AISIC and used their participation to push for expanded attention to algorithmic discrimination, labor impacts, and democratic accountability. After the Trump administration's renaming of the institute to CAISI and the apparent dropping of bias and fairness work from the priority list, several of these groups issued statements expressing concern that the United States no longer had a federal evaluation body covering the full range of AI risks.^[18]

A recurring criticism, shared across reactions, was the institute's small budget and its dependence on voluntary cooperation. Without statutory authority to compel testing, without independent compute resources, and without comprehensive AI legislation backing it, the institute's leverage was always reputational. The 2025 to 2026 DeepSeek studies and the subsequent five-lab MOU expansion changed this picture somewhat, but did not resolve the underlying funding and authority constraints.

The 2025 transition to CAISI sharpened these critiques. FedScoop observed that the renamed body's primary role appeared to be "more or less the same" as before.^[18] Others, including authors writing for the Mossavar-Rahmani Center at Harvard's Kennedy School, argued that the renaming was about "priorities, not semantics," a narrowing of scope away from the broader public-interest framing of the original AI RMF.^[30] The AI security executive order debated in May 2026, which officials likened to FDA-style pre-release review, was read by Fortune and others as a partial return to oversight ideas the administration had rejected;^[37][38] the version signed in June 2026 was voluntary and gave the lead roles to the Treasury, the NSA and CISA rather than CAISI.^[53]

Continue reading

• AAI safetyAI safety is the research and practice of preventing or reducing unacceptable harm from artificial intelligence systems.Mentioned 33 times above
• AnthropicAnthropic is an American artificial intelligence (AI) safety and research company founded in 2021 by Dario Amodei, Daniela Amodei, and other former OpenAI researchers, best known for the Claude family of large…Mentioned 24 times above
• OpenAIOpenAI is an American artificial intelligence research and deployment organization.Mentioned 21 times above
• DeepSeekDeepSeek is a Chinese artificial intelligence company based in Hangzhou. It was founded in 2023 by Liang Wenfeng, who had previously co-founded the quantitative investment firm High-Flyer.Mentioned 14 times above
• UUK AI Security InstituteThe UK AI Security Institute (AISI) is a UK government research organization (part of the Department for Science, Innovation and Technology until July 2026, when responsibility for it moved to the Cabinet…Mentioned 10 times above
• Claude (language model)Claude is a family of large language models (LLMs) developed by Anthropic, an American artificial intelligence safety and research company.Mentioned 8 times above

More from See also

• AI Safety Institutes
• AI Alignment
• Executive Order on AI
• Bletchley Declaration
• Seoul Declaration
• Frontier Model Forum
• OpenAI o1
• Responsible Scaling Policy
• Donald Trump
• Kamala Harris
• Joe Biden

Topics:AI Policy & RegulationAI Safety

References

1. ^1 ^2 ^3The White House. "Executive Order on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence." 30 October 2023. bidenwhitehouse.archives.gov/...icial-intelligence
2. ^1 ^2 ^3Deadline. "Kamala Harris To Announce U.S. AI Safety Institute As VP Attends Global Summit On The Emerging Technology." 1 November 2023. deadline.com/...artificial-intelligence-1235589191
3. ^Washington Post. "Bletchley Declaration on AI offers roadmap to limit the risks." 1 November 2023. washingtonpost.com/...ai-kamala-harris-summit
4. ^1 ^2 ^3 ^4 ^5 ^6NIST. "U.S. Commerce Secretary Gina Raimondo Announces Key Executive Leadership at U.S. AI Safety Institute." 8 February 2024. nist.gov/...-announces-key-executive-leadership-us
5. ^1 ^2 ^3 ^4 ^5 ^6NIST. "Biden-Harris Administration Announces First-Ever Consortium Dedicated to AI Safety." 8 February 2024. nist.gov/...ces-first-ever-consortium-dedicated-ai
6. ^1 ^2NIST. "U.S. Commerce Secretary Gina Raimondo Announces Expansion of U.S. AI Safety Institute Leadership Team." 16 April 2024. nist.gov/...mondo-announces-expansion-us-ai-safety
7. ^VentureBeat. "NIST staffers revolt against expected appointment of 'effective altruist' AI researcher to US AI Safety Institute." 7 March 2024. venturebeat.com/...rcher-to-us-ai-safety-institute
8. ^1 ^2 ^3 ^4NIST. "U.S. AI Safety Institute Signs Agreements Regarding AI Safety Research, Testing and Evaluation With Anthropic and OpenAI." 29 August 2024. nist.gov/...greements-regarding-ai-safety-research
9. ^1 ^2 ^3 ^4 ^5NIST. "Pre-Deployment Evaluation of Anthropic's Upgraded Claude 3.5 Sonnet." November 2024. nist.gov/...n-anthropics-upgraded-claude-35-sonnet
10. ^1 ^2 ^3 ^4 ^5NIST. "Pre-Deployment Evaluation of OpenAI's o1 Model." December 2024. nist.gov/...deployment-evaluation-openais-o1-model
11. ^1 ^2 ^3 ^4NIST. "FACT SHEET: U.S. Department of Commerce & U.S. Department of State Launch the International Network of AI Safety Institutes at Inaugural Convening in San Francisco." November 2024. nist.gov/...-department-state-launch-international
12. ^1 ^2 ^3The White House. "Removing Barriers to American Leadership in Artificial Intelligence." 23 January 2025. whitehouse.gov/...rship-in-artificial-intelligence
13. ^Bloomberg. "Head of US AI Safety Institute to Leave as Trump Shifts Course." 4 February 2025. bloomberg.com/...e-to-leave-as-trump-shifts-course
14. ^1 ^2 ^3 ^4Reuters (Jeffrey Dastin). "AI Safety Institute Director Leaves Role." 5 February 2025. insurancejournal.com/...811086
15. ^1 ^2 ^3 ^4The American Presidency Project. "Remarks by the Vice President at the Artificial Intelligence Action Summit in Paris, France." 11 February 2025. presidency.ucsb.edu/...-action-summit-paris-france
16. ^1 ^2Washington Post. "Vance pushes 'America First' AI agenda, accuses allies of overregulation." 11 February 2025. washingtonpost.com/...vance-paris-ai
17. ^1 ^2 ^3 ^4 ^5 ^6 ^7U.S. Department of Commerce. "Statement from U.S. Secretary of Commerce Howard Lutnick on Transforming the U.S. AI Safety Institute into the Pro-Innovation, Pro-Science U.S. Center for AI Standards and Innovation." 3 June 2025. commerce.gov/...-howard-lutnick-transforming-us-ai
18. ^1 ^2FedScoop. "Trump administration rebrands AI Safety Institute." June 2025. fedscoop.com/...nds-ai-safety-institute-aisi-caisi
19. Technical.ly. "US Commerce Department renames, refocuses AI Safety Institute." 2025. technical.ly/...-institute-overhaul-howard-lutnick
20. ^1 ^2 ^3 ^4NIST. "Center for Advancing Innovation and Standards for Super Intelligence (CAISSI)" (formerly Center for AI Standards and Innovation). Accessed November 2026. nist.gov/caisi
21. ^1 ^2 ^3 ^4 ^5 ^6MeriTalk. "Trump Administration Taps Chris Fall to Lead CAISI." April 2026. meritalk.com/...tion-taps-chris-fall-to-lead-caisi
22. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9NIST. "CAISI Signs Agreements Regarding Frontier AI National Security Testing With Google DeepMind, Microsoft and xAI." May 2026. web.archive.org/...er-ai-national-security-testing
23. ^U.S. Senate Committee on Commerce, Science, & Transportation. "Commerce Committee Passes Bipartisan Bill to Ensure U.S. Leads Global AI Innovation." 2024. commerce.senate.gov/...telligence-safety-institute
24. ^Mintz. "AI Provisions in Biden's FY 2025 Budget Proposal." March 2024. mintz.com/...ons-bidens-fy-2025-budget-proposal-ai
25. Wikipedia. "Artificial intelligence safety institute." en.wikipedia.org/...AI_Safety_Institute
26. ^CSIS. "The U.S. Vision for AI Safety: A Conversation with Elizabeth Kelly, Director of the U.S. AI Safety Institute." csis.org/...-kelly-director-us-ai-safety-institute
27. ^1 ^2 ^3CDO Magazine. "Ex-OpenAI Researcher Paul Christiano Named AISI's Head of AI Safety." April 2024. cdomagazine.tech/...-named-aisis-head-of-ai-safety
28. ^1 ^2CNBC. "OpenAI and Anthropic agree to let U.S. AI Safety Institute test and evaluate new models." 29 August 2024. cnbc.com/...let-us-ai-safety-institute-test-models
29. ^NIST. "International Network for Advanced AI Measurement, Evaluation, and Science Publishes Consensus Areas on Practices for Automated Evaluations." February 2026. nist.gov/...-ai-measurement-evaluation-and-science
30. ^Harvard Kennedy School Mossavar-Rahmani Center. "From Safety to Secure Innovation" and "Renaming the US AI Safety Institute Is About Priorities, Not Semantics." 2025. hks.harvard.edu/...safety-secure-innovation
31. ^1 ^2 ^3 ^4 ^5NIST. "CAISI Evaluation of DeepSeek AI Models Finds Shortcomings and Risks." 30 September 2025. nist.gov/...ai-models-finds-shortcomings-and-risks
32. ^1 ^2 ^3NIST. "Evaluation of DeepSeek AI Models" (PDF). 30 September 2025. nist.gov/...I_Evaluation_of_DeepSeek_AI_Models.pdf
33. ^1 ^2 ^3NIST. "CAISI Evaluation of DeepSeek V4 Pro." 5 May 2026. nist.gov/...caisi-evaluation-deepseek-v4-pro
34. ^Digital Watch Observatory. "DeepSeek V4 trails US frontier by eight months, according to CAISI evaluation." May 2026. dig.watch/...pseek-v4-pro-caisi-us-nist-evaluation
35. ^1 ^2 ^3Nextgov/FCW. "Commerce AI center will evaluate Google Deepmind, Microsoft and xAI models." 5 May 2026. nextgov.com/...413349
36. ^CNN Business. "Microsoft, Google and xAI will let the government test their AI models before launch." 5 May 2026. cnn.com/...ft-google-xai-government-test-ai-models
37. ^1 ^2 ^3 ^4Fortune. "Trump administration suddenly embraces AI oversight ideas it once rejected." 6 May 2026. fortune.com/...nce-rejected-anthropic-mythos-caisi
38. ^1 ^2 ^3 ^4 ^5Federal News Network. "WH 'studying' AI security executive order." May 2026. federalnewsnetwork.com/...security-executive-order
39. ^1 ^2 ^3 ^4 ^5Axios. "U.S. ramps up frontier AI testing as White House pivots toward safety." 5 May 2026. axios.com/...-ai-testing-white-house-pivots-safety
40. ^1 ^2Anthropic. "Project Glasswing: Securing critical software for the AI era." April 2026. anthropic.com/glasswing
41. ^1 ^2NPR. "How AI is getting better at finding security holes." 11 April 2026. npr.org/...asswing-ai-cybersecurity-mythos-preview
42. ^1 ^2 ^3 ^4PBS NewsHour. "Trump orders federal agencies to stop using Anthropic tech over AI safety dispute." February 2026. pbs.org/...g-anthropic-tech-over-ai-safety-dispute
43. ^Axios. "Trump officials draft plan to bring Anthropic back amid Pentagon fight." 29 April 2026. axios.com/...ropic-pentagon-ai-executive-order-gov
44. ^The Daily Signal. "Commerce Department Changes Direction on AI Safety Leader." 23 April 2026. dailysignal.com/...trump-anthropic-researcher
45. ^USAJOBS. "Member of Technical Staff, Frontier Assessment" (CAISI). 2026. ai.usajobs.gov/...856265200
46. TechRepublic. "DeepSeek AI Models Are Unsafe and Unreliable, Finds NIST-Backed Study." October 2025. techrepublic.com/...seek-security-gaps-caisi-study
47. ^1 ^2NIST. "CAISI's Assessment of Z.ai's GLM-5.3 Cyber Capabilities." 17 September 2026. nist.gov/...essment-zais-glm-53-cyber-capabilities
48. ^Cloud Security Alliance. "EO 14434 and CAISSI: Mostly a Rebrand, With One Provision to Watch." 7 October 2026. labs.cloudsecurityalliance.org/...-caissi-20261007
49. ^1 ^2 ^3ExecutiveGov. "Chris Fall Steps Down as CAISI Director, NIST Chief Raman to Serve as Acting Head." 21 July 2026. executivegov.com/...-fall-resignation-arvind-raman
50. ^1 ^2Brookings Institution. "Elham Tabassi to join Brookings as the new Director of Artificial Intelligence and Emerging Technology Initiative." 30 January 2025. brookings.edu/...nd-emerging-technology-initiative
51. ^1 ^2OpenAI. "Paul Christiano joins OpenAI Foundation Board." 9 September 2026. openai.com/...stiano-joins-openai-foundation-board
52. ^NIST. "NIST Expands AI Consortium's Scope, Calls for New Members." 29 May 2026. nist.gov/...ai-consortiums-scope-calls-new-members
53. ^1 ^2 ^3The White House. "Promoting Advanced Artificial Intelligence Innovation and Security." 2 June 2026. whitehouse.gov/...lligence-innovation-and-security

Show all 53 references