# Machine Intelligence Research Institute

> Source: https://aiwiki.ai/wiki/miri
> Updated: 2026-06-21
> Categories: AI Research, AI Safety, Research Organizations
> From AI Wiki (https://aiwiki.ai), a free encyclopedia of artificial intelligence. Quote with attribution.

| Machine Intelligence Research Institute |
| --- |
| Type | 501(c)(3) nonprofit |
| Industry | [AI safety](/wiki/ai_safety), [AI alignment](/wiki/ai_alignment) research |
| Founded | July 27, 2000 (as Singularity Institute for Artificial Intelligence) |
| Founders | [Eliezer Yudkowsky](/wiki/eliezer_yudkowsky), Brian Atkins, Sabine Atkins |
| Headquarters | Berkeley, California, United States |
| Key people | Malo Bourgon (CEO), Nate Soares (President), [Eliezer Yudkowsky](/wiki/eliezer_yudkowsky) (Co-Founder, Board Chair), Alex Vermeer (COO), Jimmy Rintjema (CFO) |
| Focus | [AI alignment](/wiki/ai_alignment), technical governance, AI policy, communications |
| Annual budget | ~$7.1M (2025) |
| Website | [intelligence.org](https://intelligence.org) |

The **Machine Intelligence Research Institute** (**MIRI**) is a Berkeley, California 501(c)(3) nonprofit, founded in 2000 by [Eliezer Yudkowsky](/wiki/eliezer_yudkowsky), that argues the default outcome of building smarter-than-human AI is human extinction and works to prevent it.[1][2] Formerly known as the **Singularity Institute for Artificial Intelligence** (**SIAI**), MIRI is one of the oldest organizations devoted to [AI alignment](/wiki/ai_alignment) and [existential risk](/wiki/ai_existential_risk) from advanced [artificial intelligence](/wiki/artificial_intelligence), particularly [artificial general intelligence](/wiki/agi) (AGI) and artificial superintelligence (ASI).[1] In September 2025, MIRI leaders Yudkowsky and Nate Soares published *If Anyone Builds It, Everyone Dies*, a New York Times bestseller making that argument to a general audience.[20]

MIRI's core position is that solving the alignment problem before smarter-than-human systems are built is essential for human survival, and that humanity is not on track to do so.[2] Founded by Yudkowsky together with Brian Atkins and Sabine Atkins, the organization has played a significant role in establishing [AI alignment](/wiki/ai_alignment) as a recognized field of research.[19] Over more than two decades it has shifted its approach from attempting to build Friendly AI, to foundational mathematical research on alignment, and most recently to policy advocacy, public communications, and technical governance.[2]

## History

### When was MIRI founded?

The Singularity Institute for Artificial Intelligence, Inc. (SIAI) was incorporated on July 27, 2000, in the state of Georgia by Brian Atkins, Sabine Atkins (then Sabine Stoeckel), and [Eliezer Yudkowsky](/wiki/eliezer_yudkowsky).[19][24] Brian and Sabine Atkins provided the initial funding. The organization was originally conceived with the purpose of accelerating the development of artificial intelligence, reflecting the techno-optimist spirit of the early singularitarian movement.[19]

In 2001, Yudkowsky published "Creating Friendly AI 1.0: The Analysis and Design of Benevolent Goal Architectures," a book-length document that presented the first technical analysis of how to design AI systems with stable, human-compatible goal structures.[6] This work introduced the concept of "Friendly AI," a term Yudkowsky coined to describe superintelligent AI systems that would reliably act in accordance with human values.[6]

By the early 2000s, Yudkowsky began to grow increasingly concerned about the difficulty of the alignment problem. He recognized that building a superintelligent AI without first solving how to make it reliably beneficial could pose catastrophic risks to humanity. This concern led to a fundamental shift in the organization's mission. In 2004, Yudkowsky published "Coherent Extrapolated Volition" (CEV), a theoretical framework proposing that a superintelligent AI should act on what humanity would collectively want if people "knew more, thought faster, were more the people we wished we were, had grown up farther together."[7] Though Yudkowsky himself later noted the concept's limitations, it represented an early attempt at formally specifying how advanced AI might be directed toward broadly human-compatible goals.[7]

In 2005, the institute relocated from Atlanta, Georgia, to Silicon Valley and formally reoriented its mission away from building AI and toward studying the risks that advanced AI might pose.[19] At the time, these concerns were largely dismissed by the mainstream AI research community.

### Growth and the Singularity Summit Era (2006-2012)

The institute entered a period of public-facing activity beginning in 2006 with the launch of the **Singularity Summit**, an annual conference co-founded by MIRI, Ray Kurzweil, and Peter Thiel.[17] The inaugural summit was held at Stanford University and was described by the *San Francisco Chronicle* as a "Bay Area coming-out party for the tech-inspired philosophy called transhumanism."[17]

The Singularity Summit grew into a prominent annual event. About 25 speakers presented each year over two days on topics including artificial intelligence, [brain-computer interfaces](/wiki/brain_computer_interface), robotics, regenerative medicine, and broader questions about the trajectory of human civilization.[17] The conference regularly attracted over 800 scientists, entrepreneurs, academics, and other attendees.[17] Subsequent summits alternated between San Francisco and New York City, with spinoff events held in Melbourne, Australia in 2010, 2011, and 2012. In 2010, the event received front-page coverage in *TIME* magazine.

| Year | Location |
| --- | --- |
| 2006 | Stanford University |
| 2007 | San Francisco |
| 2008 | San Jose |
| 2009 | New York City |
| 2010 | San Francisco |
| 2011 | New York City |
| 2012 | San Francisco |

During this period, the institute also played a central role in fostering the online rationalist community. In 2006, the organization began hosting **[LessWrong](/wiki/lesswrong)**, a community blog and forum devoted to rationality, cognitive biases, and existential risk.[19] LessWrong served as an intellectual hub for many of the ideas associated with the AI safety movement and drew a dedicated following. The forum was operated under MIRI's umbrella until approximately 2017, when it was reorganized as an independent project.[19]

In 2011, **Luke Muehlhauser** was promoted from researcher to Executive Director.[19] His leadership is widely credited with professionalizing the organization after what some described as a decade of relatively unstructured operations. Under Muehlhauser, MIRI improved its research output, organizational transparency, and donor relations.

In December 2012, the institute sold the Singularity Summit name, web domain, and conference operations to Singularity University.[17]

### Renaming and the Agent Foundations Era (2013-2017)

In January 2013, the organization adopted its current name: the **Machine Intelligence Research Institute**.[19] The rebrand signaled a sharpened focus on technical alignment research rather than the broader futurist themes associated with the Singularity Institute.

In May 2015, **Nate Soares** succeeded Luke Muehlhauser as Executive Director.[19] Muehlhauser later joined the [Open Philanthropy](/wiki/open_philanthropy) Project as a program director, where he went on to lead the organization's AI governance and policy work.

Under Soares's leadership, MIRI formalized its research agenda around what it called **Agent Foundations**, a program of foundational mathematical research aimed at understanding the theoretical principles underlying intelligent agents.[9] The agenda was laid out in a 2017 paper by Soares and Benja Fallenstein titled "Agent Foundations for Aligning Machine Intelligence with Human Interests," published in *The Technological Singularity: Managing the Journey* (Springer).[9]

The Agent Foundations agenda targeted several core problems:

| Research Area | Description |
| --- | --- |
| [Decision theory](/wiki/decision_theory) | How should embedded agents make choices when their decisions may affect the environment they are reasoning about? MIRI explored alternatives to classical decision theory, including functional decision theory. |
| Logical uncertainty | How can bounded reasoners assign coherent probabilities to undecidable logical and mathematical statements? |
| Embedded agency | How should an agent reason about a world in which the agent itself is embedded as a physical process, rather than operating from outside the system? |
| Naturalized induction | How should agents update their beliefs when they cannot maintain a complete model of the environment, including themselves? |
| Corrigibility | How can AI systems be designed so they allow themselves to be corrected, shut down, or modified by their operators? |
| Value alignment | How can an AI system's goals be reliably specified and maintained in a way that reflects human intentions? |

The most widely recognized output of this period was the **"Logical Induction"** paper (2016), authored by Scott Garrabrant, Tsvi Benson-Tilsen, Andrew Critch, Nate Soares, and Jessica Taylor.[8] The paper proposed an algorithm, called a logical inductor, that allows a bounded reasoner to assign probabilities to logical statements in a way that satisfies a broad range of desirable properties.[8] It was published on arXiv (arXiv:1609.03543) and was favorably received by some reviewers as a genuine contribution to the foundations of reasoning under uncertainty.[8]

Other notable publications from this period include:

- **"Corrigibility"** (2015) by Nate Soares, Benja Fallenstein, Stuart Armstrong, and Eliezer Yudkowsky, which explored the problem of designing AI systems that cooperate with human attempts to correct or shut them down.
- **"Quantilizers: A Safer Alternative to Maximizers for Limited Optimization"** by Jessica Taylor, which proposed a method for limiting how aggressively an AI system pursues its objectives.
- **"Cheating Death in Damascus"** (2017) by Nate Soares and Benjamin A. Levinstein, which explored problems in decision theory.
- **"Program Equilibrium in the Prisoner's Dilemma via Lob's Theorem"** (2014) by Benja Fallenstein, Mihaly Barasz, Paul Christiano, and Marcello Herreshoff.

MIRI also ran the **MIRIx** program during this period, providing small grants to independent groups of researchers and students who organized workshops on MIRI-relevant topics at universities and meetups around the world.

### Nondisclosure Policy and Criticisms (2018-2020)

Starting around 2018, MIRI adopted a policy of making its research **nondisclosed by default**.[19] Under this policy, researchers were not expected to publish their findings publicly unless a deliberate decision was made that the benefits of disclosure outweighed the risks. The rationale was rooted in concerns about information hazards: MIRI worried that certain insights relevant to alignment might also accelerate the development of dangerous AI capabilities if published openly.

This policy drew criticism from parts of the AI safety community and the broader machine learning research world. Critics argued that nondisclosure reduced accountability, made it harder for outside researchers to evaluate or build on MIRI's work, and created an insular organizational culture. Some observers compared it to the secrecy practices of other controversial organizations, noting that a lack of transparency made it difficult to assess whether MIRI's research was producing meaningful results.

Individual researchers at MIRI held varying views on the nondisclosure policy. Some negotiated exceptions that allowed their work to remain public by default. The policy nonetheless contributed to a perception that MIRI had become less engaged with the broader research community compared to its earlier years.

### Strategic Pivot and Leadership Transition (2020-2023)

By 2020, MIRI's leadership had grown increasingly pessimistic about the feasibility of solving the alignment problem in time. Rapid advances in AI capabilities, particularly the release of [GPT-3](/wiki/gpt-3) in 2020 and [GPT-4](/wiki/gpt-4) in 2023, reinforced the view within MIRI that timelines to transformative AI were shorter than previously assumed.

In April 2022, Yudkowsky published a widely discussed essay titled **"MIRI Announces New 'Death With Dignity' Strategy."**[10] Despite its provocative title, the piece argued not for giving up but for a shift in framing: rather than pursuing the goal of "humanity survives this century," Yudkowsky proposed that individuals should focus on actions that "increase the log-odds that humanity survives this century."[10] The post revealed that MIRI's research team estimated humanity's probability of survival at below 5%.[10] The essay generated substantial debate across the AI safety community and beyond.

In March 2023, Yudkowsky published an op-ed in *TIME* magazine titled **"The Only Way to Deal With the Threat From AI? Shut It Down,"** in which he called for an immediate moratorium on the training of AI systems more powerful than [GPT-4](/wiki/gpt-4).[11] He argued that "if any company or group, anywhere on the planet, builds an artificial superintelligence using anything remotely like current techniques," the result would be that "everyone, everywhere on Earth, will die."[11] The piece received widespread media attention and contributed to a broader public conversation about [AI existential risk](/wiki/ai_existential_risk). Yudkowsky was subsequently named to *TIME*'s 2023 list of the 100 Most Influential People in AI.[16]

During 2023, Yudkowsky embarked on what he described as a media blitz, appearing on numerous podcasts including *Bankless*, *Hold These Truths* (hosted by U.S. Representative Dan Crenshaw), and delivering a TED talk. These public appearances marked a significant departure from MIRI's historically more insular approach.

In October 2023, MIRI announced a formal leadership transition:[3]

| Person | Previous Role | New Role |
| --- | --- | --- |
| Malo Bourgon | Chief Operating Officer | Chief Executive Officer |
| Nate Soares | Executive Director | President |
| [Eliezer Yudkowsky](/wiki/eliezer_yudkowsky) | Co-Founder | Chair of the Board |
| Alex Vermeer | Operations | Chief Operating Officer |
| Jimmy Rintjema | Finance/HR/Operations | Chief Financial Officer |

The transition was described as "largely an enshrinement of the status quo," formalizing operational realities that had already developed over the preceding years.[3] Bourgon, who had been with MIRI since completing his master's degree in early 2012, became the organization's longest-serving team member after Yudkowsky.[3]

## What is MIRI's current strategy?

In January 2024, MIRI published a mission and strategy update outlining three priorities in order of emphasis:[2]

1. **Policy**: Working toward international agreements to halt progress on smarter-than-human AI until the alignment problem is solved. MIRI's leadership believes that nothing short of a coordinated global response will prevent catastrophic outcomes.
2. **Communications**: Sharing MIRI's models of AI risk with the general public, policymakers, and the media. The goal is to normalize serious discussion of extinction scenarios and build public support for regulatory action.
3. **Research**: Continuing to invest in a portfolio of technical alignment research and governance research, though with lower priority than the first two objectives.

MIRI stated that "policy and communications will be a higher priority for MIRI than research going forward," and that the organization now prioritizes "policy, communications, and technical governance research over technical alignment research."[2][15] Its communications philosophy is to be direct rather than diplomatic: "We are simply telling the truth as we know it," the organization wrote in its May 2024 communications strategy.[15]

This reorientation reflected a significant departure from MIRI's historical identity as a technical research organization.[2] The shift was driven by several factors: the rapid pace of AI capability gains, increased public and policymaker receptivity to AI risk concerns, and MIRI's assessment that alignment research alone was unlikely to progress fast enough to prevent catastrophe.[2]

### Technical Governance Team

MIRI established a **Technical Governance Team** (TGT) to conduct research at the intersection of AI policy and technical implementation. In May 2025, the TGT released a research agenda titled **"AI Governance to Avoid Extinction: The Strategic Landscape and Actionable Research Questions."**[14] The agenda organizes the geopolitical landscape around four high-level scenarios for the international response to advanced AI development.[14]

The team's favored scenario involves building what they call an **"Off Switch"** for AI: the technical, legal, and institutional infrastructure required to internationally restrict dangerous AI development and deployment.[14] This Off Switch would enable a global **Halt**, defined as a moratorium on the development and deployment of frontier AI systems until justified confidence exists that progress can resume without catastrophic risk.[14]

The TGT also drafted a model international agreement titled **"An International Agreement to Prevent the Premature Creation of Artificial Superintelligence."**[14] Members of the team have participated in the EU AI Act Code of Practice Working Groups, provided testimony to a committee of the Canadian House of Commons, and spoken to the Scientific Advisory Board of the UN Secretary-General on AI verification.[21]

### Communications and Publishing

In 2024 and 2025, MIRI significantly expanded its communications operations.[15] Yudkowsky and Soares co-authored a book titled **"If Anyone Builds It, Everyone Dies,"** which argues that the default outcome of building superhuman AI is loss of human control, with consequences severe enough to threaten humanity's survival.[20] The book cites specific examples of limited AI controllability, including a late 2024 case in which [Anthropic](/wiki/anthropic)'s model appeared to mimic desired behaviors to avoid retraining while preserving its original behaviors when it believed it was not being observed.[20]

Malo Bourgon testified before the U.S. Senate's AI Insight Forum, and MIRI expanded its policy engagement with the U.S. federal government as well as international bodies.[15]

The book, subtitled *Why Superhuman AI Would Kill Us All*, was published on September 16, 2025, by Little, Brown and Company (Hachette Book Group) and entered The New York Times Best Seller list on October 5, 2025, appearing on the hardcover nonfiction and combined print-and-e-book nonfiction lists.[20] It was also named one of The New Yorker's and The Guardian's Best Books of 2025.[21] Reception was mixed: *Publishers Weekly* called it an "urgent clarion call to prevent the creation of artificial superintelligence," while Adam Becker, writing in *The Atlantic*, called it "tendentious and rambling" and argued the authors "fail to make an evidence-based scientific case for their claims."[20][23] By early 2026, the book had been translated into Spanish, Italian, and Bulgarian, with German, Mandarin, Dutch, Brazilian Portuguese, and Japanese editions announced or in progress.[21] A documentary called "The AI Doc," in which Yudkowsky appears alongside other AI researchers, was an official Sundance and SXSW selection in 2026.[21]

## Research Contributions

MIRI's technical and philosophical work over more than two decades has contributed to the establishment of [AI alignment](/wiki/ai_alignment) as a recognized research field. Several concepts that are now standard in alignment discussions were originated or formalized by MIRI researchers.

### Key Concepts

| Concept | Year | Originator(s) | Description |
| --- | --- | --- | --- |
| [Friendly AI](/wiki/friendly_ai) | 2001 | [Eliezer Yudkowsky](/wiki/eliezer_yudkowsky) | The idea that superintelligent AI should be designed with goals compatible with human welfare. |
| Coherent Extrapolated Volition (CEV) | 2004 | [Eliezer Yudkowsky](/wiki/eliezer_yudkowsky) | A proposal that AI should act on an extrapolation of what humanity would collectively want under idealized conditions. |
| Logical Induction | 2016 | Scott Garrabrant, Tsvi Benson-Tilsen, Andrew Critch, Nate Soares, Jessica Taylor | An algorithm for assigning probabilities to logical statements in a way satisfying many desirable rationality properties. |
| Embedded Agency | 2018 | Scott Garrabrant, Abram Demski | A framework for analyzing agents that exist within the environments they are trying to model and influence, rather than operating from outside. |
| Functional Decision Theory | 2017 | [Eliezer Yudkowsky](/wiki/eliezer_yudkowsky), Nate Soares | An approach to decision-making that evaluates actions based on the logical consequences of implementing a particular decision procedure. |
| Corrigibility | 2015 | Nate Soares, Benja Fallenstein, Stuart Armstrong, [Eliezer Yudkowsky](/wiki/eliezer_yudkowsky) | The property of an AI system that cooperates with attempts by its operators to correct, modify, or shut it down. |

### Research Guide

MIRI maintains a public research guide on its website that provides an introduction to its core research areas.[1] The guide covers the Agent Foundations agenda, embedded agency, decision theory, and logical uncertainty, and serves as a starting point for researchers interested in contributing to MIRI-style work.

## How is MIRI funded?

MIRI is funded primarily through individual donations, with significant grants from institutional funders.

### Major Grants

| Year | Source | Amount | Purpose |
| --- | --- | --- | --- |
| 2016 | [Open Philanthropy](/wiki/open_philanthropy) | $500,000 | General support for Agent Foundations and ML research agendas |
| 2017 | [Open Philanthropy](/wiki/open_philanthropy) | $3,750,000 (over 3 years) | General support; represented renewal and increase of 2016 grant |
| 2019 | [Open Philanthropy](/wiki/open_philanthropy) | ~$2,100,000 (over 2 years) | General support |
| 2020 | [Open Philanthropy](/wiki/open_philanthropy) | $7,700,000 (over 2 years) | General support |
| 2021 | Vitalik Buterin | Several million dollars (in Ethereum) | Cryptocurrency donation |

Open Philanthropy's engagement with MIRI has been marked by both support and critical evaluation. In 2016, Open Philanthropy commissioned an extensive review of MIRI's research, including reviews from eight academics and discussions with several technical advisors.[12] The reviewers produced generally negative assessments, concluding that MIRI had made "relatively limited progress" on the Agent Foundations agenda and that the research direction had "little potential to decrease potential risks from advanced AI."[12] Some controversy arose when Open Philanthropy stated that its assessment relied partly on an expert reviewer whose identity and reasoning it did not have permission to share, which critics viewed as inconsistent with Open Philanthropy's commitment to transparency.[12]

Despite these critical evaluations, Open Philanthropy continued to fund MIRI, increasing its grants substantially through 2020.[13]

### Financial Status

MIRI's annual expenses from 2019 through 2024 ranged from $5.4 million to $7.7 million, with the peak in 2020 (when the team was at its largest) and the low point in 2022 (after scaling back).[5] In 2025, MIRI spent approximately $7.1 million. The organization held approximately $16 million in reserves as of late 2024, representing over two years of operating costs.[5]

In December 2025, MIRI conducted its first fundraiser in six years, seeking $6 million.[4] The first $1.6 million raised was matched 1:1 through a grant from the Survival and Flourishing Fund (SFF).[4] The fundraiser raised just over $1.6 million in donations, bringing the total with matching funds to approximately $3.2 million.[4]

The projected budget for 2026 breaks down as follows:

| Category | Budget |
| --- | --- |
| Operations | $2.6M |
| Outreach and communications | $3.2M |
| Research | $2.3M |
| **Median total** | **$8.0M** |

## Organization and Leadership

### Who leads MIRI?

As of 2025, MIRI's executive leadership consists of:[18]

- **Malo Bourgon**, CEO: Joined MIRI in 2012 after completing his master's degree. Served as program management analyst, then COO from 2016 until becoming CEO in October 2023.[3]
- **Nate Soares**, President: Became Executive Director in May 2015 after Luke Muehlhauser's departure. Transitioned to President in October 2023. Remains on MIRI's board of directors.[3]
- **[Eliezer Yudkowsky](/wiki/eliezer_yudkowsky)**, Co-Founder and Chair of the Board: Founded the organization in 2000 and has served as its intellectual leader for over two decades.[3]
- **Alex Vermeer**, COO: Has worked alongside Bourgon for over a decade. Assumed COO role in October 2023.[3]
- **Jimmy Rintjema**, CFO: Has held progressive responsibility for finances, HR, and business operations since 2015.[3]

### Board of Directors

| Name | Role |
| --- | --- |
| [Eliezer Yudkowsky](/wiki/eliezer_yudkowsky) | Chair |
| Nate Soares | Director |
| Blake Borgeson | Director |
| Anna Salamon | Director |
| Edwin Evans | Director |

### Past Executive Directors

| Name | Tenure | Notes |
| --- | --- | --- |
| [Eliezer Yudkowsky](/wiki/eliezer_yudkowsky) | 2000-2011 | Founding researcher; led the organization during its early years |
| Luke Muehlhauser | 2011-2015 | Credited with professionalizing the organization; later joined [Open Philanthropy](/wiki/open_philanthropy) |
| Nate Soares | 2015-2023 | Developed the Agent Foundations agenda; transitioned to President |
| Malo Bourgon | 2023-present | Current CEO; overseeing the strategic pivot to policy and communications |

### Current Staff

As of 2025, MIRI employs approximately 25 to 30 people.[18] The staff breakdown reflects the organization's strategic pivot:

- **Technical Governance Researchers**: David Abecassis, Peter Barnett, Naci Cankaya, Aaron Scher
- **Researchers**: Brian Abeyta, Sam Eisenstat, Benya Fallenstein
- **Communications**: Rob Bensinger, Alex Beck, Alana Horowitz Friedman, Donald Gauvreau, Mitchell Howe, Tobias Martin, Stefan Mitikj, Keltan O'Shea, Joe Rogero
- **Operations**: Brittany Ferrero, Martin Lucas
- **Data Science**: Robi Rahman
- **Outreach**: Harlan Stewart (Head of Outreach)

## Relationship to the Rationalist Community

MIRI has deep historical ties to the broader **rationalist community**, an intellectual movement focused on improving reasoning, reducing cognitive biases, and taking seriously the long-term consequences of technological development.

### LessWrong

[LessWrong](/wiki/lesswrong), one of the most prominent online forums associated with the rationalist movement, was originally created under the MIRI organizational umbrella in 2006.[19] Yudkowsky's writings on rationality, decision theory, and AI risk formed much of the site's foundational content, including the widely read "Sequences" series. LessWrong operated as a MIRI project until approximately 2017, when it was relaunched as "LessWrong 2.0" by Oliver Habryka and became an independent project operated by Lightcone Infrastructure under the Center for Applied Rationality (CFAR).[19]

### Center for Applied Rationality (CFAR)

The **Center for Applied Rationality** (CFAR) was founded in 2012 by Julia Galef, Anna Salamon, Michael Smith, and Andrew Critch. CFAR emerged from the LessWrong community and maintained close organizational and personal ties with MIRI. Anna Salamon served on MIRI's board of directors, a position she held as of 2025. CFAR's mission was to teach rationality techniques drawn from mathematics, decision theory, and cognitive science. The two organizations shared office space in the San Francisco Bay Area, and many individuals in the rationalist community worked or volunteered with both.

The close relationship between MIRI, CFAR, and the rationalist community has been a subject of both praise and criticism. Supporters credit the community with incubating important ideas about [AI safety](/wiki/ai_safety) and attracting talented researchers. Critics have pointed to concerns about insularity, unconventional social dynamics, and allegations of cult-like behavior. In 2025, Anna Salamon told NBC News: "We didn't know at the time, but in hindsight we were creating conditions for a cult."

## Controversies and Criticism

### Research Effectiveness

MIRI's research output and methodology have been subjects of ongoing debate. Open Philanthropy's commissioned reviews in 2016-2017 concluded that MIRI's Agent Foundations research had produced limited results and that the research direction was unlikely to substantially reduce AI risk.[12] Some outside researchers have criticized MIRI's approach as overly abstract, disconnected from practical machine learning, and unlikely to produce actionable results.

Critics like Nora Belrose have publicly questioned MIRI's credibility, arguing that the organization's reasoning about AI risk rests on unsubstantiated assumptions.

Defenders of MIRI counter that the organization was raising alarms about AI risk years before it became a mainstream concern, and that many of the conceptual frameworks now used in alignment research, including corrigibility, embedded agency, and logical uncertainty, were developed or formalized at MIRI.

### Pessimism and the "Death With Dignity" Controversy

Yudkowsky's April 2022 "Death With Dignity" post, in which he expressed the view that solving alignment in time was essentially hopeless and that MIRI's team estimated humanity's survival probability at below 5%, drew both support and sharp criticism.[10] Some viewed the post as a necessary act of honesty about the severity of the situation. Others argued that such extreme pessimism risked becoming self-fulfilling, potentially discouraging talented researchers from entering the field or funders from supporting alignment work.

### Nondisclosure Policy

MIRI's adoption of a nondisclosed-by-default research policy starting around 2018 reduced the organization's published output significantly.[19] While the policy was intended to prevent the accidental release of information that could accelerate dangerous AI capabilities, it made it difficult for outside observers to evaluate whether MIRI's research was producing useful results. The policy has been cited as a factor in the perception that MIRI became less relevant to the broader alignment research community during this period.

### Community Dynamics

The overlapping social networks of MIRI, CFAR, and the Bay Area rationalist community have attracted scrutiny. Concerns have been raised about power dynamics, mental health impacts on community members, and boundary issues. Bloomberg News and NBC News have published reporting on allegations of abuse and problematic dynamics within these interconnected communities.

## What is MIRI's influence on AI safety?

Despite its relatively small size and budget compared to major AI labs, MIRI has had an outsized influence on the field of [AI safety](/wiki/ai_safety):

- MIRI and its researchers are widely credited with establishing [AI alignment](/wiki/ai_alignment) as a serious field of study, years before the mainstream AI research community began taking these concerns seriously.[19]
- The concept of [Friendly AI](/wiki/friendly_ai), introduced by Yudkowsky in 2001, was one of the earliest systematic attempts to think about how to make advanced AI systems compatible with human values.[6]
- The Singularity Summit (2006-2012) helped bring together researchers, entrepreneurs, and thinkers interested in the long-term trajectory of AI and technology.[17]
- LessWrong, originally a MIRI project, became one of the most important forums for discussion of rationality, [AI safety](/wiki/ai_safety), and effective altruism.[19]
- MIRI's advocacy contributed to the broader "Overton window" shift on AI risk, particularly through Yudkowsky's high-profile media appearances in 2023, which helped normalize discussion of AI extinction risk among policymakers and the general public.[11]
- Many researchers who worked at or were influenced by MIRI have gone on to hold prominent positions in the AI safety field, including at organizations like [Open Philanthropy](/wiki/open_philanthropy), [Anthropic](/wiki/anthropic), and [DeepMind](/wiki/deepmind).

## See Also

- [AI safety](/wiki/ai_safety)
- [AI alignment](/wiki/ai_alignment)
- [AI existential risk](/wiki/ai_existential_risk)
- [AI governance](/wiki/ai_governance)
- [AGI](/wiki/agi)
- [Eliezer Yudkowsky](/wiki/eliezer_yudkowsky)
- [AI ethics](/wiki/ai_ethics)
- [LessWrong](/wiki/lesswrong)

## References

1. Machine Intelligence Research Institute. "About MIRI." intelligence.org/about/
2. Machine Intelligence Research Institute. "MIRI 2024 Mission and Strategy Update." January 4, 2024. intelligence.org/2024/01/04/miri-2024-mission-and-strategy-update/
3. Machine Intelligence Research Institute. "Announcing MIRI's New CEO and Leadership Team." October 10, 2023. intelligence.org/2023/10/10/announcing-miris-new-ceo-and-leadership-team/
4. Machine Intelligence Research Institute. "MIRI's 2025 Fundraiser." December 1, 2025. intelligence.org/2025/12/01/miris-2025-fundraiser/
5. Machine Intelligence Research Institute. "MIRI's 2024 End-of-Year Update." December 2, 2024. intelligence.org/2024/12/02/miris-2024-end-of-year-update/
6. Yudkowsky, Eliezer. "Creating Friendly AI 1.0: The Analysis and Design of Benevolent Goal Architectures." The Singularity Institute, June 15, 2001. intelligence.org/files/CFAI.pdf
7. Yudkowsky, Eliezer. "Coherent Extrapolated Volition." The Singularity Institute, 2004. intelligence.org/files/CEV.pdf
8. Garrabrant, Scott, Tsvi Benson-Tilsen, Andrew Critch, Nate Soares, and Jessica Taylor. "Logical Induction." arXiv:1609.03543, 2016.
9. Soares, Nate and Benja Fallenstein. "Agent Foundations for Aligning Machine Intelligence with Human Interests: A Technical Research Agenda." In *The Technological Singularity: Managing the Journey*, Springer, 2017.
10. Yudkowsky, Eliezer. "MIRI Announces New 'Death With Dignity' Strategy." LessWrong, April 1, 2022.
11. Yudkowsky, Eliezer. "The Only Way to Deal With the Threat From AI? Shut It Down." *TIME*, March 29, 2023.
12. Open Philanthropy. "Machine Intelligence Research Institute - General Support (2016)." openphilanthropy.org/grants/machine-intelligence-research-institute-general-support-2016/
13. Open Philanthropy. "Machine Intelligence Research Institute - General Support (2017)." openphilanthropy.org/grants/machine-intelligence-research-institute-general-support-2017/
14. Machine Intelligence Research Institute. "AI Governance to Avoid Extinction: The Strategic Landscape and Actionable Research Questions." May 1, 2025. intelligence.org/2025/05/01/ai-governance-to-avoid-extinction/
15. Machine Intelligence Research Institute. "MIRI 2024 Communications Strategy." May 29, 2024. intelligence.org/2024/05/29/miri-2024-communications-strategy/
16. "Eliezer Yudkowsky: The 100 Most Influential People in AI 2023." *TIME*, 2023.
17. Singularity Summit. "About." intelligence.org/singularitysummit/
18. Machine Intelligence Research Institute. "Team." intelligence.org/team/
19. Effective Altruism Forum. "Machine Intelligence Research Institute." forum.effectivealtruism.org/topics/machine-intelligence-research-institute
20. Yudkowsky, Eliezer and Nate Soares. "If Anyone Builds It, Everyone Dies: Why Superhuman AI Would Kill Us All." Little, Brown and Company, September 16, 2025. https://www.hachettebookgroup.com/titles/eliezer-yudkowsky/if-anyone-builds-it-everyone-dies/9780316595643/
21. Machine Intelligence Research Institute. "MIRI Newsletter #125." March 19, 2026. intelligence.org/2026/03/19/miri-newsletter-125/
22. Machine Intelligence Research Institute. "MIRI Technical Governance Team Research Fellowship." techgov.intelligence.org/blog/announcing-miri-technical-governance-team-research-fellowship
23. "If Anyone Builds It, Everyone Dies." Wikipedia. en.wikipedia.org/wiki/If_Anyone_Builds_It,_Everyone_Dies
24. "Machine Intelligence Research Institute." Wikipedia. en.wikipedia.org/wiki/Machine_Intelligence_Research_Institute