Future of Life Institute
Last reviewed
Sources
No citations yet
Review status
Needs citations
Revision
v4 ยท 4,322 words
Improve this article
Add missing citations, update stale details, or suggest a clearer explanation.
Suggest edit
CtrlK
Last reviewed
Sources
No citations yet
Review status
Needs citations
Revision
v4 ยท 4,322 words
Add missing citations, update stale details, or suggest a clearer explanation.
The Future of Life Institute (FLI) is a United States nonprofit research and outreach organization that works on existential and large-scale risks from transformative technologies, with a focus on advanced ai safety and the governance of frontier artificial intelligence.[1] It was founded in March 2014 by Massachusetts Institute of Technology cosmologist Max Tegmark, Skype co-founder Jaan Tallinn, University of California Santa Cruz physicist Anthony Aguirre, Tufts University postdoctoral researcher Meia Chita-Tegmark, and google deepmind research scientist Viktoriya Krakovna.[1][2] FLI rose to broad public prominence through a series of open-letter campaigns: the 2015 Puerto Rico research-priorities letter that accompanied its first conference, the 2017 Asilomar AI Principles, the March 2023 "Pause Giant AI Experiments" letter signed by elon musk, yoshua bengio, Steve Wozniak, and more than 30,000 others, and the October 2025 "Statement on Superintelligence".[3][4][5][20] Since December 2024 it has also published the AI Safety Index, an independent letter-graded report card scoring the leading frontier AI companies; in the Summer 2025 edition six expert reviewers graded seven companies and awarded a top overall grade of only C+, to Anthropic.[35][36] The institute runs a grants program for AI safety research seeded by a 2015 donation of $10 million from Elon Musk, manages the Future of Life Award for unsung contributors to humanity's survival, and advocates on AI policy in the United States, the European Union, and at the United Nations.[6][7][8] As of late 2025, Max Tegmark is president and Anthony Aguirre is executive director, with the organization registered in California and active offices in Brussels and Washington, D.C.[9][2]
FLI was incorporated in early 2014 by Tegmark, Tallinn, Aguirre, Chita-Tegmark, and Krakovna, with a stated mission of "steering transformative technology towards benefiting life and away from extreme large-scale risks."[1][2] Its launch event, held at MIT on 24 May 2014, was a public panel titled The Future of Technology: Benefits and Risks, moderated by the actor Alan Alda. Panelists included synthetic biologist George Church, geneticist Ting Wu, economist Andrew McAfee, Nobel laureate physicist Frank Wilczek, and Tallinn.[1] Tegmark, the lead co-founder, was already a professor of physics at MIT and would become widely known the following year through the 2015 Puerto Rico conference and the 2017 book Life 3.0: Being Human in the Age of Artificial Intelligence, which expanded on FLI's research agenda.[10] Aguirre, the institute's executive director, is the Faggin Presidential Professor for the Physics of Information at UC Santa Cruz and a long-time collaborator of Tegmark on cosmology and the Foundational Questions Institute (FQXi).[9]
FLI organized its first technical conference, The Future of AI: Opportunities and Challenges, in San Juan, Puerto Rico, on 2 to 5 January 2015.[7] Conference participants drafted and signed an open letter titled "Research Priorities for Robust and Beneficial Artificial Intelligence", authored by Stuart Russell, Daniel Dewey, and Tegmark, that called for technical work on AI alignment, verification, validation, and societal impact.[7][11] On 15 January 2015, Elon Musk announced a $10 million donation to FLI to fund a global research grant program based on the letter's priorities.[6][12] In October 2015 FLI named 37 research teams selected from nearly 300 applicants for the first round of the program, distributing roughly $7 million in initial awards.[13] open philanthropy later contributed an additional roughly $1.2 million to expand the same request-for-proposals beyond the Musk funding.[14]
FLI's second flagship conference, Beneficial AI 2017, was held at the Asilomar Conference Grounds in Pacific Grove, California, from 5 to 8 January 2017.[15] More than 100 researchers and practitioners in AI, economics, law, ethics, and philosophy participated, including geoffrey hinton, Yoshua Bengio, stuart russell, Demis Hassabis, Yann LeCun, Sam Harris, and Musk.[15] The conference produced the Asilomar AI Principles, a set of 23 guidelines for AI research divided into three categories (research issues, ethics and values, and longer-term issues) that were subsequently signed by 1,797 AI and robotics researchers and 3,923 other endorsers.[4][15]
In November 2017 FLI and Stuart Russell released Slaughterbots, a short dramatic film depicting swarms of palm-sized autonomous drones using face-recognition software to assassinate people, screened at the United Nations Convention on Certain Conventional Weapons in Geneva.[16][17] The film attracted millions of views and became a recurring reference in policy debates over lethal autonomous weapons.[16] FLI followed it with a 2018 "Lethal Autonomous Weapons Pledge", signed by AI researchers and companies who committed not to develop or deploy weapons that select and engage targets without meaningful human control, and with a 2021 sequel film, Slaughterbots: if human kill().[2][16]
On 22 March 2023, less than a week after OpenAI released gpt-4, FLI published an open letter titled "Pause Giant AI Experiments" that called on "all AI labs to immediately pause for at least 6 months the training of AI systems more powerful than GPT-4."[5][18] The letter argued that contemporary systems were becoming "human-competitive at general tasks" and that further training should be paused until shared safety protocols, third-party audits, and governance institutions were in place.[5] Within days, the letter had thousands of signatories; by 2024 it carried more than 30,000 signatures, including Bengio, Russell, Musk, Wozniak, Yuval Noah Harari, Emad Mostaque, Tristan Harris, Andrew Yang, Tegmark, and Daron Acemoglu.[5][18] No major frontier laboratory paused its training programs, but the letter intensified policy debate over advanced AI and was widely cited in subsequent legislative and regulatory discussions in the United States and the European Union.[18][19]
On 11 December 2024, FLI published the inaugural AI Safety Index, an independent report card in which a panel of seven AI and governance experts graded the safety practices of six leading general-purpose AI companies (Anthropic, Google DeepMind, Meta, OpenAI, xAI, and Zhipu AI) across six categories: risk assessment, current harms, safety frameworks, existential safety, governance and accountability, and transparency.[35][37] Anthropic received the top overall grade of C, while every other company scored D+ or lower and Meta received an F.[37] FLI continued the project on a roughly semiannual cadence. The Summer 2025 edition, released on 17 July 2025, was assessed by six independent reviewers, including stuart russell, Dylan Hadfield-Menell, David Krueger, Tegan Maharaj, Jessica Newman, and Sneha Revanur, and graded seven companies; Anthropic again led with a C+, ahead of OpenAI (C) and Google DeepMind (C-), with xAI and Meta at D and DeepSeek and Zhipu AI at F.[36][38] Russell summarized the panel's concern that "we are spending hundreds of billions of dollars to create superintelligent AI systems over which we will inevitably lose control," while Tegmark argued that "self-regulation simply isn't working, and the only solution is legally binding safety standards like we have for medicine, food and airplanes."[38] The Winter 2025 edition, published in early December 2025, again awarded no company higher than a C+ and gave every company a grade no better than D on existential safety, with the report concluding that the firms are racing toward agi without an actionable plan to keep smarter-than-human systems controllable.[39]
On 22 October 2025, FLI published a follow-up "Statement on Superintelligence", a one-sentence open letter calling for "a prohibition on the development of superintelligence, not lifted before there is broad scientific consensus that it will be done safely and controllably, and strong public buy-in."[20][21] Early signatories included Hinton, Bengio, Russell, Wozniak, Virgin Group founder Richard Branson, former Irish president Mary Robinson, and conservative commentator Steve Bannon.[21][22] By early 2026 the statement had attracted more than 130,000 signatures, making it the largest single AI safety petition coordinated by FLI to date.[21]
Max Tegmark is a Swedish-American cosmologist and machine-learning researcher who has been a professor of physics at MIT since 2004 and is FLI's president.[10][9] Trained as a cosmologist, Tegmark shifted his public research focus toward AI safety in the 2010s and authored Life 3.0: Being Human in the Age of Artificial Intelligence (Knopf, 2017), which discusses long-term outcomes of AI development and frames FLI's broader research agenda.[10] Tegmark co-founded the Foundational Questions Institute (FQXi) with Aguirre in 2005 and helped organize the Puerto Rico and Asilomar conferences.[7][9]
Jaan Tallinn is an Estonian software engineer and a founding engineer of Kazaa and Skype's peer-to-peer file-sharing and voice telephony technologies; he later became an active philanthropist on AI risk.[1] In addition to co-founding FLI, Tallinn co-founded the Cambridge Centre for the Study of Existential Risk and the Survival and Flourishing Fund, two other organizations focused on catastrophic and existential risk from advanced technology.[1]
Anthony Aguirre is a U.S. physicist and Faggin Presidential Professor for the Physics of Information at UC Santa Cruz; he is FLI's executive director and treasurer of its board.[9][2] He co-founded the Foundational Questions Institute with Tegmark in 2005, has authored a popular book on cosmology, and runs the forecasting platform Metaculus, which FLI has used for AI forecasting work.[9]
Meia Chita-Tegmark is a social psychology researcher who completed postdoctoral work at Tufts University on human-robot interaction and is married to Max Tegmark.[1][2] She is a co-founder of FLI and a board member, and she also co-founded the AI Health nonprofit focused on AI for medicine.[2]
Viktoriya Krakovna is a Latvian-Canadian research scientist at Google DeepMind, where she works on AI safety, specification gaming, and emergent goal-directed behavior in learned agents.[1] She co-founded FLI in 2014 while a Harvard statistics PhD student and serves on its board.[2]
FLI's programmatic work spans research grants, policy advocacy, public communications, an independent safety benchmark, and an awards program. Its principal program areas are listed below.
FLI maintains policy teams in the United States and Europe that produce position papers, model legislation, and government testimony on advanced AI.[23] In the European Union, FLI's Brussels office has been one of the most active civil-society participants in the eu ai act process, publishing a series of position papers in 2022 and 2023 that pushed for the inclusion of general-purpose and foundation models in the regulation; FLI claimed credit for several provisions related to systemic-risk models in the final text.[23][24] In Washington, FLI has filed comments on the 2023 Biden administration Executive Order on AI and on subsequent rulemakings by the National Institute of Standards and Technology, and it has briefed congressional offices on third-party evaluations and reporting requirements for frontier developers.[23] The institute also publishes a manifesto for each five-year European Commission cycle outlining priorities for AI Office staffing, the AI Liability Directive, and standards work.[25]
FLI's first grant round, funded by the 2015 Musk donation, awarded $7 million across 37 projects on topics including value alignment, verification, economic impact, and policy.[13] A 2018 second round, also funded primarily by Musk with support from Open Philanthropy, awarded a smaller batch of grants focused on artificial general intelligence safety.[2] In July 2021, after receiving a large unconditional donation from Ethereum co-founder Vitalik Buterin, FLI announced a $25 million existential risk reduction grant program, including the Vitalik Buterin PhD and postdoctoral fellowships on existential safety research in AI.[26][27] Buterin's gift was a large transfer of meme cryptocurrency tokens (including Shiba Inu) that the institute converted to roughly $665 million in fiat over the following two years, according to an EU transparency filing made public in 2023.[28]
The AI Safety Index is FLI's independent grading project for frontier AI developers, first published in December 2024 and updated roughly twice a year.[35][36] An independent panel of AI and governance experts scores each company across six safety and security domains and issues letter grades from A to F, with the explicit aim of creating competitive pressure for stronger safety practices and giving policymakers a comparative baseline.[36][37] No company has ever scored above a C+ overall, and in every edition through Winter 2025 no company has scored better than a D on existential safety, a finding FLI uses to argue that voluntary self-regulation is inadequate and that legally binding standards are needed.[37][38][39]
FLI has run an autonomous-weapons program since 2015. The program produced the 2015 open letter on autonomous weapons signed by AI and robotics researchers including Hawking, Musk, and Wozniak, the Slaughterbots short films of 2017 and 2021, the 2018 Lethal Autonomous Weapons Pledge, and ongoing engagement with the UN Convention on Certain Conventional Weapons in Geneva.[16][17] The autonomousweapons.org website is operated as a campaign hub for the program, including educational resources and the Slaughterbots films.[17]
The Future of Life Award, established in 2017, is given each year to "an individual who, without having received much recognition at the time, has helped make today dramatically better than it could have been."[8] Each laureate (or their family, if posthumous) receives a $50,000 prize.[8] The inaugural 2017 award went to the family of Soviet naval officer Vasili Arkhipov, whose 1962 refusal to authorize a nuclear torpedo during the Cuban Missile Crisis is credited with averting nuclear war.[8] Subsequent recipients include Stanislav Petrov (2018) for his 1983 decision not to forward a false Soviet missile-warning alert, molecular biologist Matthew Meselson (2019) for work that led to the international ban on biological weapons, and a 2020 award to William Foege and other figures credited with the eradication of smallpox.[8]
FLI also operates the Future of Life podcast, an educational platform on existential risk and AI, and a Futures program studying possible long-term scenarios; it has supported public-comment campaigns on deepfakes and on the use of AI in nuclear command and control.[2][25]
The institute is best known to general audiences for a series of high-profile open letters.
The Puerto Rico letter, "Research Priorities for Robust and Beneficial Artificial Intelligence", was drafted by Russell, Dewey, and Tegmark and signed by Hawking, Musk, Wozniak, and thousands of AI researchers and others; it was the first widely publicized statement from the modern AI research community calling for sustained safety research on advanced systems.[7][11] The full research-priorities document was later posted to arXiv and used by FLI as the basis for its first grant request for proposals.[11]
The 23 Asilomar AI Principles emerged from the January 2017 conference and were posted as an open letter that attracted signatures from nearly 1,800 AI and robotics researchers, including Bengio, LeCun, Hassabis, Hinton, and others, along with figures such as Musk and the late Stephen Hawking.[15][4] The principles cover research goals, safety research funding, the avoidance of an arms race in lethal autonomous weapons, value alignment, and long-term governance.[4]
The March 2023 letter is the most discussed of FLI's letters. It opens with the claim that "AI systems with human-competitive intelligence can pose profound risks to society and humanity" and asks for an industry-wide pause of at least six months on training systems "more powerful than GPT-4."[5] High-profile signatories include Bengio, Russell, Musk, Wozniak, Harari, Stability AI chief executive Emad Mostaque, Tristan Harris of the Center for Humane Technology, Andrew Yang, Tegmark, and MIT economist Daron Acemoglu.[5][18] Coverage in Reuters, the Financial Times, Wired, IEEE Spectrum, and other outlets framed the letter as a turning point in mainstream political attention to advanced AI risks.[19][29]
The letter also drew sharp criticism. AI ethics researchers Timnit Gebru, Emily Bender, Angelina McMillan-Major, and Margaret Mitchell, authors of the "Stochastic Parrots" paper that the letter cited, said the letter co-opted their work and that its "fearmongering and AI hype" steered attention away from existing harms such as data exploitation and discrimination.[18][30] Eliezer Yudkowsky argued in Time that the letter did not go far enough and called for an indefinite, internationally enforced moratorium.[31] OpenAI chief executive Sam Altman declined to sign and disputed the letter's framing of GPT-5 development, while figures including Bill Gates and Reid Hoffman publicly rejected the proposal for a fixed-window pause.[18] Commentators from accelerationist and effective-altruism backgrounds also clashed over the letter's premises, with some viewing it as too timid and others as alarmist.[18][30]
The October 2025 "Statement on Superintelligence" framed FLI's position more sharply, calling for a conditional ban on the development of any system that could surpass humans across most cognitive tasks until safety and societal consent are established.[20] By January 2026 it had more than 130,000 signatures, with prominent supporters including Hinton, Bengio, Russell, Wozniak, Branson, Robinson, Bannon, the Duke and Duchess of Sussex, and several Nobel laureates.[21][22]
The Asilomar AI Principles are FLI's most cited normative document. Adopted at the Beneficial AI 2017 conference at Asilomar from 5 to 8 January 2017 and posted publicly on 11 August 2017, the principles comprise 23 statements grouped into three categories.[4][15] The research issues section (five principles) covers research goals, funding, science and policy links, research culture, and avoidance of races to the bottom on safety.[4] The ethics and values section (13 principles) covers safety, failure transparency, judicial transparency, responsibility, value alignment, human values, personal privacy, liberty and privacy, shared benefit, shared prosperity, human control, non-subversion of civic processes, and avoidance of an arms race in lethal autonomous weapons.[4] The longer-term issues section (five principles) covers capability caution, importance of advanced AI as a profound change, risks proportional to expected impact, recursive self-improvement, and "common good" framing of superintelligence.[4] The principles have been cited in subsequent governance frameworks such as the OECD AI Principles and the G20 AI guidelines.[15]
FLI is a 501(c)(3) nonprofit registered in the United States and reports its finances on annual IRS Form 990 filings and on its own website.[32] Major contributions have come from Elon Musk, who provided $10 million in 2015 to launch the AI safety grants program; Open Philanthropy, which provided multiple general-support and AI risk-reduction grants in 2017 and 2018 totaling about $1.9 million on top of its contributions to the Musk RFP; and Vitalik Buterin, whose 2021 donation of meme tokens generated roughly $665 million in fiat after liquidation, recorded in an EU transparency filing made public in 2023.[6][13][14][28] Additional donors include the Berkeley Existential Risk Initiative, the Survival and Flourishing Fund, and individual contributors.[32] The Buterin donation made FLI one of the largest funders in the AI safety ecosystem and enabled it to bankroll the Vitalik Buterin PhD and postdoctoral fellowships, additional grant rounds, and the expansion of its policy team in Brussels and Washington.[26][27]
Max Tegmark is FLI's president and the chair of its board.[9][2] Anthony Aguirre is executive director and board treasurer, responsible for day-to-day operations and the institute's research direction.[9][2] Other named board members include Meia Chita-Tegmark, Viktoriya Krakovna, and Jaan Tallinn, with various external advisers drawn from the AI safety and existential risk research community.[2] As of 2024 the institute reports a staff of more than 30 across policy, communications, grantmaking, and operations, with offices or staff in Campbell, California; Washington, D.C.; Brussels; and London.[9][2] FLI's policy team is led by Mark Brakel as director of policy, with Risto Uuk leading EU policy research and Hamza Tariq Chaudhry leading U.S. policy as of 2024.[23][24]
FLI has received extensive coverage in mainstream and trade press. The 2015 Puerto Rico conference and Musk donation were covered by TechCrunch, CNN Money, IEEE Spectrum, Robohub, and other outlets, which positioned FLI as one of the first organizations to convene mainstream AI researchers around safety topics.[6][12][13] Slaughterbots was covered by IEEE Spectrum, CNN, and the Bulletin of the Atomic Scientists, with Stuart Russell using the film to argue in Congressional testimony that autonomous weapons should be classified as weapons of mass destruction.[16][33] The 2023 pause letter received front-page coverage in the New York Times, Washington Post, Financial Times, Wired, and others, and was cited in U.S. Senate hearings and in EU debates over the AI Act.[19][29][24] The AI Safety Index has likewise been covered by IEEE Spectrum, Axios, and others as a closely watched, if contested, scorecard of frontier-lab safety practices.[36][39]
Reception has not been uniformly positive. FLI has been criticized by AI ethics researchers, including Gebru and the co-authors of the "Stochastic Parrots" paper, for centering long-term and existential risks at what they argue is the expense of near-term algorithmic harms.[30] Some observers have linked FLI to the broader effective altruism and longtermism movements and questioned the philosophical grounding of its grant-making priorities.[30][28] Others, particularly in accelerationist circles, have framed FLI's letters as overstating risks from current systems; Sam Altman, Bill Gates, and Reid Hoffman publicly declined to sign the 2023 pause letter while engaging with its concerns.[18] FLI has responded with explanatory FAQs and follow-up articles defending the letters' framing and arguing that long-term and near-term harms are complementary, not competing, priorities.[34]