CtrlK
The US AI Safety Institute (USAISI or US AISI), since June 2025 the Center for AI Standards and Innovation (CAISI), is a research and evaluation body housed within the National Institute of Standards and Technology (NIST), an agency of the United States Department of Commerce. The institute was created in late 2023 as the American counterpart to the UK AI Safety Institute, with a mandate to develop measurement science for evaluating advanced artificial intelligence systems, conduct pre-deployment testing of frontier models, and coordinate with international peers through the network of national AI safety institutes.
Its inaugural director, Elizabeth Kelly, was appointed in February 2024 and led the institute through its first year of operation, including the August 2024 memoranda of understanding with Anthropic and OpenAI and the November 2024 inaugural convening of the International Network of AI Safety Institutes in San Francisco. Kelly stepped down on 4 February 2025, two weeks after President Donald Trump returned to office, revoked Executive Order 14110, and signed Executive Order 14179 redirecting federal AI policy from risk mitigation toward innovation and global competitiveness. On 3 June 2025 Commerce Secretary Howard Lutnick announced that the institute would be reorganized as the Center for AI Standards and Innovation, dropping the word "safety" from its title and reorienting its mission around national security risks and what the administration termed "pro-innovation" standards. As of mid-2026, CAISI continues to operate within NIST under a new director, Chris Fall, and has signed pre-deployment testing agreements with Google DeepMind, Microsoft, and xAI; Anthropic remains outside the new framework.
The US AI Safety Institute is the product of two converging strands of policy. The first was the rapid acceleration of frontier model capabilities during 2022 and 2023, particularly with the public release of ChatGPT in November 2022 and the introduction of GPT-4 in March 2023. By mid-2023, the Biden administration had already secured a series of voluntary commitments from leading AI developers, including red-teaming pledges, watermarking research, and information sharing on model weights and security practices. These commitments were widely seen as insufficient given the pace of capability growth, and Congress had not yet passed any comprehensive AI legislation.
The second strand was international. In April 2023 the United Kingdom government announced the Frontier AI Taskforce, an internal team focused on building government capacity for the technical evaluation of frontier AI models, and signaled that it would host the world's first major intergovernmental conference on frontier AI risks at Bletchley Park in November 2023. UK officials lobbied other G7 governments to make parallel investments in evaluation capacity. The United States response was to create its own evaluation body, anchored at NIST, which had decades of experience publishing voluntary technical standards and had already produced the AI Risk Management Framework (AI RMF 1.0) in January 2023.
NIST's existing AI work made it the natural home for the new institute. The agency's National Cybersecurity Center of Excellence, its Computer Security Resource Center, and its Information Technology Laboratory had all built relationships with industry on standards-setting, and Elham Tabassi, then chief AI advisor at NIST, had led the development of the AI RMF and was already a recognized figure in the international AI policy community.
Vice President Kamala Harris, attending the AI Safety Summit at Bletchley Park on 1 November 2023, announced the formation of the US AI Safety Institute in a speech at the US embassy in London. Harris's announcement was timed to coincide with Prime Minister Rishi Sunak's launch of the UK AI Safety Institute and the signing of the Bletchley Declaration. Her speech framed the institute's role as developing technical guidance for regulators on issues including content authentication, watermarking of AI-generated outputs, algorithmic discrimination, and privacy.
Harris's announcement built on Executive Order 14110, "Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence," which President Joe Biden had signed two days earlier on 30 October 2023. Section 4.1(a) of EO 14110 directed the Secretary of Commerce, acting through NIST, to establish guidelines and best practices for the development and deployment of safe, secure, and trustworthy AI systems and to develop "companion resources" to the AI RMF. The executive order also instructed the Secretary of Commerce to designate developers of dual-use foundation models above a 10^26 floating-point operation training threshold as subject to ongoing reporting requirements under the Defense Production Act, and to direct NIST to develop standards for red-teaming, content authentication, and security testing of these models.
The institute was formally stood up over the following months. On 8 February 2024, Commerce Secretary Gina Raimondo announced its initial executive leadership and the launch of the AI Safety Institute Consortium. Raimondo's announcement at the time described the institute as having a mandate to operate as the federal government's primary contact point for industry on AI safety testing and to coordinate with international partners.
Until June 2025, the institute's mission as published on its NIST landing page was "to advance the science, practice, and adoption of AI safety across the spectrum of risks, including those to national security, public safety, and individual rights." Its activities clustered around three operational priorities.
The first was developing measurement and evaluation methodology. The institute drew on NIST's expertise in metrology to design empirical tests of model capabilities in domains of public concern: cybersecurity, biological and chemical misuse risk, software autonomy, and the robustness of model safeguards against jailbreaks and adversarial use. It worked closely with industry, the Frontier Model Forum, and academic groups to translate informal capability assessments into reproducible benchmarks.
The second was conducting pre-deployment evaluations of frontier models from US developers. Unlike the UK AI Safety Institute, which had access to Crown computing resources and was structured as a startup-like operational unit, the US institute relied on memoranda of understanding with private developers, NIST's existing technical staff, and contractor support. Its first formal evaluations were conducted jointly with the UK institute under a US-UK partnership agreement signed in April 2024.
The third was producing voluntary standards and guidance. NIST's existing publication apparatus, including its Special Publication and Internal Report series, gave the institute a mature pathway for releasing draft profiles, guidance documents, and risk assessments for public comment. In July 2024 NIST published the Generative AI Profile of the AI RMF (NIST AI 600-1), which became the principal voluntary reference document for industry compliance with the executive order's safety expectations.
The institute was structured as an office within NIST's headquarters operations rather than as an independent agency or laboratory. It did not have regulatory authority. Its findings could inform other agencies (including the Department of Homeland Security, the Department of Defense, the Bureau of Industry and Security, and the Federal Trade Commission), but the institute itself could not compel disclosure, mandate testing, or fine non-compliant developers. This structural limitation, repeatedly identified by outside commentators, reflected both the absence of comprehensive AI legislation and the politically contested nature of the executive order that authorized the institute.
The institute's executive team was assembled in two waves during early 2024.
Elizabeth Kelly was named inaugural director on 7 February 2024. Before joining NIST she had served as a special assistant to President Biden for economic policy at the National Economic Council, and she was one of the principal drafters of Executive Order 14110. Her appointment signaled that the administration intended the institute to operate at the intersection of technical evaluation and economic policy. Kelly held the role until 4 February 2025, when she announced her departure in a LinkedIn post that did not specify her next position. Reuters and Bloomberg reported the departure as "leaving the institute's direction in question" given the new administration's stated opposition to the executive order that had authorized her work.
Elham Tabassi was named the institute's first chief technology officer in the same February 2024 announcement. Tabassi had led the development of the AI Risk Management Framework and was a long-tenured NIST scientist. She remained in her role through the 2025 transition and continues to serve in technical leadership at CAISI.
In April 2024, Secretary Raimondo announced an expanded leadership team that brought five new senior figures into the institute.
| Role | Name | Background |
|---|---|---|
| Director | Elizabeth Kelly | Former White House special assistant for economic policy; principal drafter of EO 14110 |
| Chief Technology Officer | Elham Tabassi | Long-tenured NIST scientist; lead author of the AI Risk Management Framework |
| Head of AI Safety | Paul Christiano | Former head of language model alignment at OpenAI; founder of the Alignment Research Center |
| Chief Vision Officer | Adam Russell | Former director of the AI Division at USC's Information Sciences Institute; former DARPA program manager |
| Acting Chief Operating Officer and Chief of Staff | Mara Quintero Campbell | Former deputy chief operating officer at the Commerce Department's Economic Development Administration |
| Senior Advisor | Rob Reich | Stanford political scientist; co-author of "System Error" |
| Head of International Engagement | Mark Latonero | Former lead of human rights and AI policy work at the UN and World Economic Forum |
The most controversial appointment was Christiano's. As founder of the Alignment Research Center, where he had pioneered work on reinforcement learning from human feedback and on capability evaluations of frontier models, Christiano was widely respected by AI safety researchers but was also publicly associated with effective altruism and the long-termist position that loss-of-control risks from advanced AI represented a central category of concern. VentureBeat reported in March 2024 that NIST staff had circulated internal protests over the planned appointment, with some threatening to resign on grounds that Christiano's intellectual associations would compromise the institute's perceived neutrality. Raimondo proceeded with the appointment in April 2024, and Christiano remained in role through the 2025 transition.
After Kelly's departure in February 2025, the directorship sat vacant for over a year. Day-to-day operations were managed by Tabassi and a rotating set of acting senior officials while the Trump administration completed its review of the executive order's implementation activities. On 28 April 2026, the Commerce Department announced that Chris Fall, a former Department of Energy official from the first Trump administration with a background in scientific research administration, would lead the renamed CAISI. Fall's first public statement framed CAISI's mission as "independent, rigorous measurement science" focused on national security implications of frontier AI.
On 8 February 2024, alongside Kelly's appointment, Secretary Raimondo announced the AI Safety Institute Consortium (AISIC), described as the first US government consortium dedicated specifically to AI safety. The consortium was organized under NIST's Cooperative Research and Development Agreement (CRADA) framework, which allows NIST to enter formal collaboration with industry and academic partners on standards-setting work.
The consortium launched with more than 200 participating organizations and grew to over 280 members by the end of 2024, making it among the largest standards consortia in any domain. Members spanned major AI developers (Anthropic, Google, Microsoft, OpenAI, Meta, Cohere, IBM, Amazon Web Services), enterprise software firms (Salesforce, Cisco, Adobe, Workday, Notion), cloud providers (Oracle, NVIDIA, Hewlett Packard Enterprise), academic and research institutions (Stanford, MIT, Carnegie Mellon, the Federation of American Scientists, the Center for Democracy and Technology), and industry associations.
The consortium's working groups focused on five technical areas:
| Working Group | Focus |
|---|---|
| 1: Risk Management for Generative AI | Adapting the AI RMF to generative AI use cases |
| 2: Synthetic Content | Developing standards for content authentication and watermarking |
| 3: Capability Evaluations | Methodology for measuring model capabilities in security-relevant domains |
| 4: Red-teaming | Standards for adversarial testing of frontier models |
| 5: Safety and Security | Operational practices for safe AI deployment |
Under Kelly, the consortium served as a vehicle for translating the institute's research priorities into industry-recognized practice. Working Groups 2 and 3 produced public draft documents during 2024, and the consortium held its first plenary meeting in early 2025 to review progress and set 2025 priorities. After the June 2025 reorganization, the consortium continued to operate, though several civil-society members have publicly noted reduced engagement on bias, equity, and transparency topics that had figured prominently in the original AI RMF.
The institute's most operationally distinctive activity was the pre-deployment evaluation of frontier models. Because NIST cannot compel access, all evaluations have proceeded under voluntary memoranda of understanding with developers.
On 29 August 2024, the institute announced its first formal MOUs with Anthropic and OpenAI. Each agreement established a framework for the institute to receive access to major new models from the two companies prior to and following their public release, to conduct collaborative research on capability and safety evaluation methodology, and to provide feedback on potential safety improvements. The MOUs also explicitly contemplated coordination with the UK AI Safety Institute, reflecting the joint testing partnership the two governments had announced in April 2024.
The substantive details of the agreements were not made public, but the institute confirmed that pre-deployment access was central. OpenAI CEO Sam Altman publicly endorsed pre-release testing in an X post the same day. Anthropic CEO Dario Amodei wrote in a December 2024 post that the agreement was modeled on Anthropic's responsible scaling policy and gave the institute access to research on safeguard robustness and dangerous capability evaluations.
The principal evaluations conducted by the institute, in most cases jointly with the UK institute, are summarized below.
| Date | Model | Co-evaluator | Domains tested |
|---|---|---|---|
| October 2024 | Anthropic Claude 3.5 Sonnet (upgraded) | UK AISI | Cybersecurity, biology, software/AI development, safeguard robustness |
| November 2024 | OpenAI o1 (preview) | UK AISI | Cybersecurity, biology, software/AI development |
| December 2024 | OpenAI o1 (release version) | UK AISI | Cybersecurity, biology, software/AI development |
The Claude 3.5 Sonnet evaluation, published as a joint US-UK report in November 2024, found that the upgraded Sonnet solved 32.5 percent of the 40 publicly available cybersecurity challenges in the US institute's test suite, compared with a 35 percent solve rate from the best reference model. UK colleagues found a 36 percent solve rate on a different suite of 47 challenges (15 public, 32 private) at apprentice-level difficulty, exceeding the best reference model's 29 percent. On safeguard robustness, the UK institute found that publicly available and privately developed jailbreaks could routinely circumvent the Sonnet 3.5 safeguards as evaluated. Both institutes flagged biology benchmarks as the most uncertain area, since claims of useful uplift to malicious actors require subject-matter judgment that automated evaluations cannot provide.
The joint evaluation of OpenAI's o1, released as a NIST publication in December 2024, was the first formal joint evaluation of an OpenAI o1 model and the first major collaborative output of the US-UK partnership. The institutes reported that o1 performed roughly on par with reference models across cybersecurity, biology, and software development domains, with the notable exception of additional capabilities in cryptography-related cybersecurity tasks. The model solved 36 percent of apprentice-level cybersecurity challenges in the UK suite, against 46 percent for the best reference model. The institutes' joint write-up emphasized methodological caveats: tests were conducted under tight time constraints, with finite resources, and the domains assessed represented only a subset of conceivable concerns.
The institute did not publish a formal pre-deployment evaluation of Anthropic's Claude 3.5 Opus (later released as Claude 3.7 Sonnet and beyond) before the change of administration. Several Anthropic releases of 2025 were evaluated through Anthropic's own responsible scaling policy and via the UK institute, but no further joint US-UK reports were issued under the AISI nameplate after the December 2024 o1 evaluation.
The institute was repeatedly described by both supporters and critics as underfunded relative to its mandate. Initial appropriations under the Fiscal Year 2024 Consolidated Appropriations Act, enacted in March 2024, included approximately $10 million dedicated to standing up the institute. NIST secured a further $10 million through a one-time Technology Modernization Fund allocation in mid-2024. By comparison, the UK AI Safety Institute operated with annual funding of around £66 million ($83 million) and access to over £1.5 billion in public computing resources.
| Fiscal year | Appropriations | Source |
|---|---|---|
| FY2024 | ~$10 million dedicated + ~$10 million one-time TMF | Consolidated Appropriations Act 2024 |
| FY2025 (proposed) | ~$50 million for AI safety activities at NIST | Biden FY2025 budget request |
| FY2025 (enacted) | Continuing resolution at FY2024 levels | Congressional impasse |
| FY2026 (proposed under Trump) | Reduced (specific amount not public) | Trump administration request |
Biden's FY2025 budget proposal, submitted in March 2024, requested approximately $50 million for AI safety activities at NIST, including an expanded AI Safety Institute, but Congress did not enact a full FY2025 appropriations bill before the change of administration, leaving the institute on continuing resolution at FY2024 levels. Senators Maria Cantwell, Todd Young, John Hickenlooper, and Marsha Blackburn's bipartisan Future of AI Innovation Act, introduced in April 2024 and reintroduced in 2026, would have codified the institute in statute and authorized increased funding, but the bill did not become law in either Congress.
The political context for the institute changed sharply on 20 January 2025, when Trump revoked Executive Order 14110 within hours of taking office. Three days later, on 23 January 2025, Trump signed Executive Order 14179, "Removing Barriers to American Leadership in Artificial Intelligence," which directed senior White House officials to review all actions taken under the prior order and to identify those "inconsistent with, or presenting obstacles to" a new pro-innovation AI policy. EO 14179 instructed agencies to revise or rescind such actions within 180 days.
Kelly's departure on 4 February 2025 came at the start of that 180-day review period. In her LinkedIn announcement she expressed continued confidence in the institute's mission. Bloomberg, Reuters, and the Wall Street Journal reported the departure as the most consequential personnel change to date in federal AI policy under the new administration. The directorship remained vacant through the spring and summer of 2025.
On 11 February 2025, Vice President JD Vance delivered a keynote address at the AI Action Summit in Paris that explicitly distanced the new administration from the Bletchley-era safety framing. Vance opened his speech by saying he was "not here to talk about AI safety," but "about AI opportunity." He criticized European-style precautionary regulation, signaled that the United States would oppose foreign rules constraining American AI firms, and outlined four pillars of administration AI policy: mitigating worker displacement, avoiding heavy regulation, working with allies on shared standards, and ensuring AI was free from "ideological bias." The United States and the United Kingdom both declined to sign the Paris summit's joint communique, marking a public divergence from the Bletchley and Seoul process.
On 3 June 2025, Commerce Secretary Howard Lutnick announced that the US AI Safety Institute would be reorganized as the Center for AI Standards and Innovation (CAISI). The announcement framed CAISI as "pro-innovation" and "pro-science" and stated that the renamed center would serve as "industry's primary point of contact within the U.S. government to facilitate testing and collaborative research related to harnessing and securing the potential of commercial AI systems." Lutnick said in his statement, "For far too long, censorship and regulations have been used under the guise of national security. Innovators will no longer be limited by these standards."
The reorganization preserved CAISI within NIST and left the AISIC consortium in place, but it narrowed the priority focus to demonstrable national security risks (cybersecurity, biosecurity, chemical weapons-related uplift, and what Lutnick described as "malign foreign influence" from adversaries' AI systems). The mission document removed earlier language on algorithmic discrimination, equity, and content provenance for AI-generated media. Reporting in Technical.ly and FedScoop indicated significant staff turnover during 2025, with several senior researchers leaving for industry positions. Christiano remained in role through the transition. Tabassi continued as chief AI advisor and CAISI's senior technical official.
On 28 April 2026, the Commerce Department confirmed Chris Fall as CAISI's director. Fall, who had previously led the Department of Energy's Office of Science and ARPA-E during the first Trump administration, was framed as a "measurement science" leader rather than an AI policy specialist. His confirmation was followed in early May 2026 by the announcement of new pre-deployment testing agreements between CAISI and Google DeepMind, Microsoft, and xAI, modeled on the 2024 Anthropic and OpenAI MOUs but explicitly framed as national security testing rather than safety evaluation. Anthropic, which had become embroiled in a separate dispute with the Pentagon over guardrails on military uses of Claude, did not sign a CAISI agreement and at the time of writing remained outside the renamed framework. OpenAI's status was ambiguous: the company had signaled continued cooperation with CAISI but had not publicly transitioned its 2024 MOU into a CAISI-branded agreement.
From its earliest months, the US AISI was framed as the American hub of a coordinated international response to frontier AI risk, with the UK as its principal partner. In April 2024, US and UK officials announced a bilateral partnership agreement under which the two institutes would conduct at least one joint pre-deployment evaluation per year, share research methodology, and coordinate on capability and safeguard testing. The first joint deliverables under that agreement were the November 2024 Claude 3.5 Sonnet evaluation and the December 2024 OpenAI o1 evaluation.
At the AI Seoul Summit in May 2024, ten countries plus the European Union signed the Seoul Statement of Intent toward International Cooperation on AI Safety Science, formally launching the International Network of AI Safety Institutes. The network was conceived as a permanent forum for coordinating evaluation methodology, pooling research, and sharing findings with member governments. The United States hosted the network's inaugural in-person convening in San Francisco on 20 and 21 November 2024, with delegations from Australia, Canada, the European Union, France, Japan, Kenya, the Republic of Korea, Singapore, the United Kingdom, and the United States. The convening produced a joint mission statement, an $11 million pooled commitment to research on synthetic content, the network's first multilateral testing exercise, and a joint statement on risk assessments for advanced AI systems.
The Seoul process was the high-water mark of network-style international cooperation. The Paris AI Action Summit in February 2025 marked a public split: Vance's speech, the US and UK refusal to sign the communique, and the renaming of the UK institute to the UK AI Security Institute the following week (announced by Technology Secretary Peter Kyle at the Munich Security Conference on 14 February 2025) all signaled that the original safety-first framing was no longer the dominant axis of cooperation. Under CAISI, US participation in the network has continued but with explicit emphasis on national security applications and on countering what the administration describes as "authoritarian influence" in AI standards-setting.
In February 2026, NIST published a consensus document on best practices for automated evaluations developed by the international network, which had been renamed the International Network for Advanced AI Measurement, Evaluation, and Science (INAMES) by the UK institute earlier that year. The renaming was less universally adopted in the United States than in the United Kingdom, and CAISI publications in 2026 generally retained the older "International Network of AI Safety Institutes" branding for cross-government communications.
The US institute operated under structural and political constraints that distinguished it sharply from its UK peer. The UK institute was, in the description of its own founding documents, "a startup in government," with significant operational autonomy from the broader UK civil service, its own £66 million annual budget, and access to public computing resources at a scale not available to most government bodies. The US institute, by contrast, was an office within NIST, dependent on annual congressional appropriations, with a budget roughly an order of magnitude smaller in its first year, and without independent access to compute.
| Dimension | US AI Safety Institute (later CAISI) | UK AI Safety Institute (later UK AISI) |
|---|---|---|
| Parent body | NIST (Commerce Department) | DSIT (Cabinet Office until 2025) |
| Founded | November 2023 announced; February 2024 stood up | November 2023 |
| Founding director | Elizabeth Kelly | Ian Hogarth (Chair) |
| First year budget | ~$10 million (FY2024) | £100 million (Frontier AI Taskforce, FY23/24) |
| Annual budget by 2025 | ~$10-20 million effective | ~£66 million |
| Structure | Office within standards agency | Operationally autonomous, startup-style |
| Regulatory authority | None | None |
| Primary mode | Voluntary standards, MOU-based testing | MOU-based testing, in-house research |
| Compute access | Limited; dependent on partners | National AI Research Resource, Isambard-AI |
| Pre-deployment evaluations published | Joint with UK (Sonnet 3.5, o1) | 30+ models; many with US AISI |
| Renamed | June 2025 to CAISI | February 2025 to UK AI Security Institute |
In substantive output, the UK institute was the more productive partner. By the end of 2025 it had published more than 30 model evaluations, the open-source Inspect framework, the ControlArena platform for AI control research, and the Frontier AI Trends Report drawing on its full two-year evaluation dataset. The US institute's published evaluation output was largely confined to the joint reports with the UK and to the AI RMF Generative AI Profile. This asymmetry reflected differences in budget, political support, and institutional autonomy, not in technical talent: Kelly, Tabassi, Christiano, and Adam Russell were widely regarded as a strong leadership team, and the AISIC consortium gave the institute deep informal connections to industry expertise.
Reception of the institute can be sorted into three broad reactions: enthusiastic support from the AI safety and AI alignment research communities, qualified support from industry, and critical reactions from civil society groups concerned about scope and capture.
Researchers in the alignment community welcomed the institute as a long-overdue federal investment in technical evaluation capacity. Christiano's appointment in particular was read as a signal that the institute would take seriously the loss-of-control risks that his Alignment Research Center had emphasized. Yoshua Bengio, Geoffrey Hinton, and a number of the signatories of the May 2023 Statement on AI Risk publicly endorsed the institute's creation.
Industry response was generally positive but measured. The AISIC's rapid growth to over 280 members reflected real industry interest in voluntary standards as an alternative to regulation, and major developers were willing to sign MOUs giving the institute pre-deployment access. At the same time, executives at Anthropic, OpenAI, Google DeepMind, and Microsoft publicly argued that the institute's mandate should be "pro-innovation" and should not become a backdoor to regulation. The Business Software Alliance and similar industry associations welcomed the June 2025 transformation to CAISI on those grounds.
Civil society responses were more critical. The Center for Democracy and Technology, the AI Now Institute, the Federation of American Scientists, and the World Privacy Forum were active members of the AISIC and used their participation to push for expanded attention to algorithmic discrimination, labor impacts, and democratic accountability. After the Trump administration's renaming of the institute to CAISI and the apparent dropping of bias and fairness work from the priority list, several of these groups issued statements expressing concern that the United States no longer had a federal evaluation body covering the full range of AI risks.
A recurring criticism, shared across reactions, was the institute's small budget and its dependence on voluntary cooperation. Without statutory authority to compel testing, without independent compute resources, and without comprehensive AI legislation backing it, the institute's leverage was always reputational. The institute's published joint evaluations with the UK were widely cited and well regarded, but the cadence of US-led publications was much lower than originally hoped.
The 2025 transition to CAISI sharpened these critiques. Some observers, including the Mossavar-Rahmani Center for Business and Government at Harvard's Kennedy School, characterized the renaming as primarily semantic, arguing that the underlying technical work on cybersecurity, CBRN risk, and AI capability evaluation continued largely intact. Others, including the Institute for AI Policy and Strategy, viewed the shift as a meaningful narrowing of scope away from the broader public-interest framing of the original AI RMF. As of mid-2026, the question of whether CAISI's narrower mandate represents continuity or rupture remains contested.