Holden Karnofsky
Last reviewed
May 3, 2026
Sources
No citations yet
Review status
Needs citations
Revision
v1 ยท 3,655 words
Improve this article
Add missing citations, update stale details, or suggest a clearer explanation.
Suggest edit
CtrlK
Last reviewed
May 3, 2026
Sources
No citations yet
Review status
Needs citations
Revision
v1 ยท 3,655 words
Add missing citations, update stale details, or suggest a clearer explanation.
Holden G. Karnofsky is an American philanthropist, nonprofit executive, and writer on artificial intelligence, best known as the co-founder of the charity evaluator GiveWell and the grantmaking organization Open Philanthropy. He served as Open Philanthropy's CEO and later co-CEO from its founding in 2014 until 2023, when he stepped back from leadership to focus on AI safety. Since January 2025 he has been a member of the technical staff at Anthropic, where he works on the company's Responsible Scaling Policy and other aspects of preparing for advanced AI. Karnofsky is also influential as a writer through his blog Cold Takes, particularly the "Most Important Century" essay series (2021), which argues that the development of transformative AI could make the 21st century the most consequential period in human history.
Karnofsky is closely associated with the effective altruism movement and the field of AI safety. His grantmaking at Open Philanthropy directed hundreds of millions of dollars toward existential risk from AI, biosecurity, animal welfare, and global health, and he was an early funder of several of the labs and research groups that now define the alignment field. He is married to Daniela Amodei, president and co-founder of Anthropic, and was a former roommate of her brother Dario Amodei.
| Field | Detail |
|---|---|
| Born | 1980 or 1981 (United States) |
| Alma mater | Harvard University (A.B. in Social Studies, 2003) |
| Known for | Co-founding GiveWell (2007); co-founding Open Philanthropy (2014); the "Most Important Century" series (2021); work on AI risk |
| Employer | Anthropic (since January 2025) |
| Previous roles | Analyst at Bridgewater Associates; co-founder and co-executive director of GiveWell; CEO and co-CEO of Open Philanthropy; visiting scholar at the Carnegie Endowment for International Peace |
| Spouse | Daniela Amodei, married August 2017 |
| Personal blog | Cold Takes (cold-takes.com) |
| Movements | Effective altruism, AI safety, longtermism |
Karnofsky grew up in the United States and attended Stevenson High School in Lincolnshire, Illinois, where he was named a National Merit Scholar in 1999. He went on to Harvard University, graduating in 2003 with an A.B. in social studies. As an undergraduate he was a member of the Harvard Lampoon. He has spoken in interviews about being drawn to questions of how to do good effectively from a young age, an interest that would later shape his career in philanthropy.
After Harvard, Karnofsky joined Bridgewater Associates, the Westport, Connecticut hedge fund founded by Ray Dalio. At Bridgewater he met Elie Hassenfeld, the colleague who would later become his co-founder. The two found themselves frustrated when they tried to research charities to give to using the same kind of evidence-based methodology Bridgewater used for investments. Charities, they discovered, mostly published feel-good marketing material rather than the kind of impact data that would let a donor compare options.
In 2006 Karnofsky and Hassenfeld started a small charity club at Bridgewater. Several colleagues pooled donations and the group set out to research where the money could do the most good. The work consumed a lot of their time, and the group's findings convinced them that there was a serious gap in the philanthropy market.
In mid-2007, Karnofsky and Hassenfeld left Bridgewater to start GiveWell as a full-time project. They raised about $300,000 in seed funding from former colleagues and incorporated a nonprofit called The Clear Fund, which would direct donations to the charities GiveWell identified as the most cost-effective. The original mission was narrow: produce rigorous, public, transparent reviews of a small number of charities working on causes where impact could actually be measured, especially global health and global poverty interventions.
GiveWell's evaluations focused on a handful of programs that had strong evidence of saving or improving lives at low cost per person, such as malaria nets, deworming pills, and direct cash transfers. The model proved popular with donors who wanted to know exactly what their money would do. Annual money moved through GiveWell-recommended charities grew from roughly $1.6 million in 2010 to about $110 million in 2015, and has continued to grow since.
GiveWell's early years included a public misstep. In December 2007, Karnofsky was caught using sock-puppet accounts on the discussion site MetaFilter, asking a question about charity evaluators under one identity and then answering it (favorably for GiveWell) under his own name without disclosing his role. The board investigated, fined Karnofsky and Hassenfeld $5,000 each, and demoted Karnofsky from his executive director position for a period. He apologized at length on the GiveWell blog. The episode became a touchstone case in nonprofit ethics and was later cited as an example of how the organization's commitment to transparency was tested early.
Karnofsky stayed deeply involved in GiveWell's research and leadership through the early 2010s. He served as vice chair of the GiveWell board until 2023.
Around 2011, Karnofsky and Hassenfeld began conversations with Cari Tuna and her husband Dustin Moskovitz, the Facebook co-founder who had become a major philanthropist after leaving the company. Tuna and Moskovitz had committed to giving away the majority of their fortune through their foundation, Good Ventures, and were looking for ways to allocate large sums effectively. The conversation led to GiveWell Labs, an internal research project at GiveWell focused on more speculative areas of giving where impact was harder to measure but potentially much larger, such as policy advocacy, scientific research, and reduction of global catastrophic risks.
GiveWell Labs grew into the Open Philanthropy Project, formally launched in 2014 as a partnership between GiveWell and Good Ventures. The new initiative was structured to take more risks than GiveWell's main research and to fund causes that did not fit GiveWell's strict evidence-based criteria, including AI safety, biosecurity, criminal justice reform, farm animal welfare, and scientific research. Karnofsky led the project from the start and became its CEO.
In 2017, Open Philanthropy formally separated from GiveWell to become an independent organization, with Karnofsky as CEO. Over the next several years, Open Philanthropy became one of the most consequential funders of the AI safety field, distributing tens of millions of dollars per year to organizations including the Machine Intelligence Research Institute, the Future of Humanity Institute at Oxford, the Center for Human-Compatible AI at Berkeley, 80,000 Hours, and others. Open Philanthropy was also an early investor in OpenAI (a $30 million grant in 2017) and later supported the founding team that became Anthropic.
In mid-2021, Alexander Berger, who had joined GiveWell as a research analyst in 2011 and led Open Philanthropy's global health and wellbeing work, was promoted to co-CEO. From 2021 onward Karnofsky and Berger ran Open Philanthropy together, with Berger overseeing the global health and wellbeing portfolio and Karnofsky focusing on the longtermist portfolio, including AI safety.
In February 2023, Karnofsky announced that he was taking a leave of absence from Open Philanthropy to work directly on AI safety, citing his belief that the field was approaching a period of unprecedented urgency. Berger continued as the day-to-day leader of the organization. On 29 April 2024, Karnofsky announced that he was leaving Open Philanthropy to become a visiting scholar at the Carnegie Endowment for International Peace, citing his desire to move away from grantmaking and the awkwardness of being so closely connected to AI funding decisions while married to the president of Anthropic. Berger became sole CEO. Karnofsky remained on Open Philanthropy's board.
In 2025 Open Philanthropy renamed itself Coefficient Giving.
From April 2024, Karnofsky served as a visiting scholar at the Carnegie Endowment for International Peace, a Washington D.C. foreign policy think tank. His Carnegie work focused on international security risks from advanced AI: which AI capabilities might increase global catastrophic risk, how to identify early warning signs through capability evaluations, and what protective measures (including information security) major AI labs and governments needed to put in place. During his Carnegie tenure he published essays on responsible scaling policies as a pragmatic approach to managing risk, including a piece titled "Developing AI Risk Management With the Same Ambition and Urgency as AI Products."
Karnofsky also became affiliated with the Berkman Klein Center for Internet and Society at Harvard during this period.
Karnofsky joined Anthropic in January 2025 as a member of the technical staff. The hire was not formally announced by the company; it was first reported by Sharon Goldman at Fortune on 13 February 2025, after Karnofsky updated his LinkedIn profile. He reports to Chief Science Officer Jared Kaplan and works on Anthropic's Responsible Scaling Policy (RSP), the set of internal protocols that governs how the company evaluates frontier model risks and decides whether new capability levels are safe to deploy. He is also involved in broader safety planning for the possibility of much more advanced AI systems.
The Anthropic role placed Karnofsky inside a company led by his wife (Daniela Amodei is president and co-founder) and his former roommate (Dario Amodei is CEO and co-founder). Anthropic and outside commentators have noted the obvious conflict-of-interest concerns raised by the family connections combined with Open Philanthropy's prior funding of the company; Karnofsky's earlier exit from grantmaking at Open Philanthropy was framed in part as a response to those concerns.
| Years | Role | Organization |
|---|---|---|
| 2003-2007 | Analyst | Bridgewater Associates |
| 2007-2014 | Co-founder, co-executive director | GiveWell |
| 2014-2021 | CEO | Open Philanthropy |
| 2021-2023 | Co-CEO (with Alexander Berger) | Open Philanthropy |
| 2023-2024 | Director of AI Strategy (on leave from co-CEO) | Open Philanthropy |
| 2024-2025 | Visiting scholar | Carnegie Endowment for International Peace |
| Since 2024 | Affiliate | Berkman Klein Center for Internet and Society, Harvard |
| Since January 2025 | Member of technical staff | Anthropic |
| Until 2023 | Vice chair (board) | GiveWell |
| Ongoing | Board member | Open Philanthropy (now Coefficient Giving) |
Karnofsky writes mostly through Cold Takes, the personal blog he launched in mid-2021 at cold-takes.com. He has not published a print book; the major Cold Takes series have been compiled as free PDFs and Kindle e-books. Before Cold Takes, much of his writing appeared on the GiveWell and Open Philanthropy blogs and in essays such as "Radical Empathy" (2017) and "My current impressions on career choice for longtermists" (2021).
The "Most Important Century" series, published on Cold Takes starting on 23 September 2021, is Karnofsky's best-known piece of writing. The argument runs roughly as follows. The long-run future of humanity could be radically different from the present, with technology eventually enabling civilizations that span the galaxy. Advances in AI, especially the development of systems that can automate scientific and technological progress, could compress the timeline to that future down to decades rather than millennia. Most of the relevant AI looks like it will be developed within the 21st century. We are therefore living in a window of time that may turn out to be uniquely consequential for the entire long-run trajectory of intelligent life, and we are not prepared.
The series consists of more than a dozen blog posts, including:
| Post | Topic |
|---|---|
| All Possible Views About Humanity's Future Are Wild | Why every consistent picture of the future implies something strange |
| This Can't Go On | The argument that current rates of economic growth cannot continue indefinitely |
| Forecasting Transformative AI: What Kind of AI? | Defining transformative AI and introducing PASTA |
| Forecasting Transformative AI: Biological Anchors | Compute-based forecasts of when transformative AI might arrive |
| AI Timelines: Where the Arguments, and the "Experts," Stand | Survey of forecasts |
| Digital People Would Be An Even Bigger Deal | Implications of digital minds |
| The Duplicator | A thought experiment on instant cloning and economic explosion |
| Why AI alignment could be hard with modern deep learning | Why current ML methods may not produce safe systems |
| Making the Best of the Most Important Century | What individuals, governments, and labs can do |
| Call to Vigilance | The closing essay |
The series was discussed extensively on the 80,000 Hours podcast and on The Ezra Klein Show, and is widely cited in the effective altruism and AI safety communities. It is available as a free PDF, a Kindle e-book, and a podcast audio version.
One of the central concepts introduced in "Most Important Century" is PASTA, which stands for Process for Automating Scientific and Technological Advancement. PASTA refers to AI systems (a single model, or a collection of them) that can automate essentially all of the human activities needed to keep scientific and technological progress going: reading papers, generating hypotheses, designing experiments, designing manufacturing processes, writing code, and so on.
Karnofsky uses PASTA rather than the more common term "AGI" to focus the argument on a narrower and more behavior-based criterion: a system does not need to match humans on every dimension of cognition. It just needs to be able to do the work that drives science and technology forward. If such systems exist, the rate of progress could in principle become limited by physical resources and compute rather than by the supply of human researchers. The result is what Karnofsky calls explosive science: a feedback loop where AI systems improve the technology that makes AI systems better, leading to a rapid run-up of capabilities. Karnofsky argues that PASTA is more likely than not to be developed within the 21st century, with a non-trivial probability (more than 10%) of it appearing within a couple of decades. He emphasizes that PASTA systems could have objectives of their own that are difficult to detect and control, which is the core of his case for taking AI alignment seriously now.
Cold Takes covers futurism, quantitative macro history, AI risk, and epistemology. Beyond the "Most Important Century" series, the blog hosts a number of other essay sequences, including:
| Series | Focus |
|---|---|
| Implications of Most Important Century | Practical applications of the framework |
| AI Safety Is Hard To Measure | Why standard ML benchmarks miss alignment problems |
| How to Make the Best of the Most Important Century | Career and policy advice |
| Has Life Gotten Better? | Long-run history of human wellbeing |
| Future-Proof Ethics | Trying to write down ethical principles that are robust to civilizational change |
| Cold Links | Short link roundups |
Individual posts that have circulated widely include "How we could stumble into AI catastrophe" (January 2023), "Racing through a minefield: The AI deployment problem," "Nearcast-based deployment problem analysis," and "AI Safety Seems Hard to Measure." The blog tone is deliberately careful and hedge-laden; the title is a joke about the genre of confident takes ("hot takes") that dominates online commentary.
Before Cold Takes, Karnofsky published extensively on the GiveWell and Open Philanthropy blogs. Notable posts include "Big Impact vs. Big Promises" (Stanford Social Innovation Review, 2014), "Radical Empathy" (2017), which argues for extending moral consideration to all beings capable of welfare including animals and possibly future digital minds, and a 2021 essay titled "My current impressions on career choice for longtermists," which became a frequently referenced text in the effective altruism career advice ecosystem.
Karnofsky is one of the most prominent figures associated with effective altruism, the social movement that tries to use evidence and reason to figure out how to benefit others as much as possible. GiveWell predated the formal EA movement but became one of its anchor institutions; Open Philanthropy was for many years the largest funder of EA-aligned organizations, including 80,000 Hours, the Centre for Effective Altruism, and Giving What We Can. Open Philanthropy under Karnofsky was also the largest single funder of AI safety research worldwide for much of the 2010s and early 2020s.
Karnofsky's relationship with the broader EA brand has been more complicated since 2022, when Sam Bankman-Fried, who had publicly identified with EA and committed large sums to AI safety and pandemic preparedness work, was arrested for fraud at FTX. Karnofsky has written publicly about lessons drawn from that episode and about the limits of relying on individual donors. He has also distanced his framing from "effective altruism" as a label, while continuing to share most of its substantive commitments around using evidence to allocate resources to the highest-impact causes.
Karnofsky's views on AI risk have evolved over more than a decade. In the early 2010s he was publicly skeptical of some of the more confident claims about superintelligent AI made by figures such as Eliezer Yudkowsky and the Machine Intelligence Research Institute, and his 2012 critique "Thoughts on the Singularity Institute (SI)" became a widely discussed exchange in the early AI safety field. Over time, partly through Open Philanthropy's research and partly through the rapid progress of large language models in the late 2010s and early 2020s, his views shifted toward taking the risk of catastrophic outcomes from advanced AI very seriously.
By the time he wrote "Most Important Century" in 2021, Karnofsky was arguing that the development of transformative AI within the lifetimes of people alive today was a real possibility and that humanity was badly under-prepared. His specific concerns include misalignment (advanced systems pursuing goals their developers did not intend and cannot easily detect), misuse (the deliberate use of advanced AI by states or other actors to acquire decisive military or political advantage), and the concentration of economic and political power that could result from controlling such systems. He has been a public proponent of so-called responsible scaling policies, voluntary commitments by frontier AI labs to evaluate model capabilities for dangerous behavior at each scale-up and to pause deployment if specific risk thresholds are crossed. His current work at Anthropic builds directly on this approach.
Karnofsky tends to write in a deliberately undramatic register and to give explicit probability ranges where he can. He has repeatedly pushed back against framings of AI risk that he sees as overconfident in either direction, and his essays often spend considerable time on what he calls "weak point" objections to his own arguments.
Karnofsky married Daniela Amodei in August 2017. The two were friends well before then through Karnofsky's college friendship with Dario Amodei, Daniela's older brother, and Karnofsky and Dario were former roommates. Daniela later co-founded Anthropic with Dario and several other former OpenAI staff in 2021, and serves as the company's president. The family connection has been the subject of recurring scrutiny because Open Philanthropy under Karnofsky was one of Anthropic's earliest financial backers and because Karnofsky has been involved in AI policy work that bears on Anthropic's commercial interests.
Karnofsky has identified himself as Jewish and has spoken in interviews about his interest in synagogue communities and in cause-area framings drawn from Jewish ethics. He lives in the San Francisco Bay Area.
| Year | Title | Venue |
|---|---|---|
| 2014 | Big Impact vs. Big Promises | Stanford Social Innovation Review |
| 2017 | Radical Empathy | Open Philanthropy |
| 2021 | The "Most Important Century" series | Cold Takes |
| 2021 | My current impressions on career choice for longtermists | EA Forum |
| 2022 | AI Safety Seems Hard to Measure | Cold Takes |
| 2022 | Why AI alignment could be hard with modern deep learning | Cold Takes |
| 2023 | How we could stumble into AI catastrophe | Cold Takes |
| 2023 | Racing through a minefield: The AI deployment problem | Cold Takes |
| 2023 | Nearcast-based "deployment problem" analysis | Cold Takes |
| 2024 | Developing AI Risk Management With the Same Ambition and Urgency as AI Products | Carnegie Endowment for International Peace |
Long-form interviews include four appearances on the 80,000 Hours podcast (2018 on Open Philanthropy and grantmaking; 2021 on the Most Important Century; 2023 on AI takeover scenarios; and a longer 2024 conversation on concrete AI safety opportunities at frontier AI companies), and an appearance on The Ezra Klein Show in October 2021.