Conjecture (AI Safety Lab)
Last reviewed
Sources
23 citations
Review status
Source-backed
Revision
v3 ยท 3,979 words
Improve this article
Add missing citations, update stale details, or suggest a clearer explanation.
Suggest edit
CtrlK
Last reviewed
Sources
23 citations
Review status
Source-backed
Revision
v3 ยท 3,979 words
Add missing citations, update stale details, or suggest a clearer explanation.
Conjecture is a London-based artificial intelligence safety research company founded in March 2022 by Connor Leahy, Sid Black, and Gabriel Alfour, all alumni of EleutherAI. The company is best known for its "Cognitive Emulation" (CoEm) research agenda, which aims to build "predictably boundable systems, not directly aligned AGIs," and for "The Compendium," an October 2024 essay arguing that humanity faces extinction risk from the race to artificial general intelligence (AGI).[1][2] Conjecture raised roughly $10 million in 2022 from investors including Nat Friedman and Daniel Gross, and was one of the most outspoken voices in the high-existential risk ("x-risk") camp of AI safety before its leadership announced a wind-down in March 2026.[3][4]
Conjecture occupied a distinctive position within the AI safety ecosystem. Unlike research-focused peers such as FAR.AI or the corporate labs of OpenAI, Anthropic, and DeepMind, Conjecture combined for-profit alignment research with high-visibility public messaging on catastrophic AI risk. The company's leadership argued that frontier development was outpacing humanity's ability to control it, and over time Conjecture's emphasis shifted from technical AI alignment work toward direct civic and policy engagement. In March 2026, CEO Connor Leahy announced that Conjecture's chapter was drawing to a close and that he would join ControlAI as its US Director, marking the effective wind-down of the company.[3][4]
Conjecture is a for-profit AI safety startup that pursued two parallel goals: a technical research program aimed at building controllable AI, and a public-advocacy program warning that the race to AGI poses a civilizational, potentially extinction-level, risk. Its signature technical bet was Cognitive Emulation (CoEm), an approach to designing AI systems out of interpretable, capability-bounded components rather than aligning a single opaque end-to-end model. Its signature public output was The Compendium, a long-form argument for AI extinction risk written for non-technical readers.[1][2]
The company was unusual in being structured as a venture-backed for-profit while pursuing a safety-first mission. It incorporated commercially so that it could move quickly, raise capital, and avoid the constraints of academic or nonprofit funding cycles, while explicitly committing to channel its work toward alignment outcomes.[7] Conjecture summarized its founding ethos, in Leahy's later retrospective, as "solve technical AI safety, or die trying."[4]
Conjecture was incorporated as Conjecture Ltd in late 2021 and publicly launched in March 2022. The founding announcement, "We Are Conjecture, A New Alignment Research Startup," was posted on the LessWrong forum on 8 April 2022 by Connor Leahy, Sid Black, and Gabriel Alfour, who outlined a mission to "scale applied AI alignment research."[5] All three founders had been active in the volunteer-driven EleutherAI community, which itself had been founded in July 2020.[6]
The original core team grew quickly. By mid-2022 Conjecture had hired around two dozen researchers and engineers, many recruited from EleutherAI, the broader Effective Altruism community, and independent alignment research circles. Notable senior staff during the company's lifetime included Adam Shimi (alignment epistemology and the Refine incubator), Andrea Miotti (Head of Strategy and Governance), and Chris Scammell (Chief Operating Officer).[7][8]
From the outset the company distinguished itself by combining commercial structure with an existential-risk-driven research program: it incorporated as a for-profit so that it could move quickly, raise capital, and avoid the constraints of academic or nonprofit funding cycles, while explicitly committing to channel its work toward alignment outcomes.[7]
| Fact | Detail |
|---|---|
| Founded | March 2022 (incorporated late 2021) |
| Founders | Connor Leahy, Sid Black, Gabriel Alfour |
| Headquarters | London, United Kingdom |
| Origin | EleutherAI alumni community (founded July 2020) |
| Funding | ~$10 million (2022), primarily venture equity |
| Signature research | Cognitive Emulation (CoEm), introduced February 2023 |
| Signature publication | The Compendium (31 October 2024) |
| Wind-down announced | March 2026 |
Conjecture's most public-facing figure, Connor Leahy, is a German-American AI researcher born in 1998. He attended the Technical University of Munich from 2017 to 2020 but left without a formal degree, and rose to prominence in 2019 after reverse-engineering OpenAI's then-withheld GPT-2 model from his student room. In July 2020 he co-founded EleutherAI with Sid Black and Leo Gao, originally as a Discord server organized to replicate GPT-3.[9]
During Leahy's tenure as a co-lead of EleutherAI, the group produced several of the earliest open-source large language models and datasets, including The Pile (December 2020), GPT-Neo (March 2021), GPT-J-6B (June 2021), and GPT-NeoX-20B (February 2022). At its release, GPT-J-6B was the largest openly available GPT-3-style model. Leahy's transition from EleutherAI to Conjecture coincided with a publicly stated shift in his views on the appropriate level of openness for frontier model research, with Leahy later arguing that capability progress had outpaced alignment progress to a degree that made unrestricted open-sourcing risky.[9][10]
Leahy served as Conjecture's CEO from its founding in March 2022 until its wind-down in 2026. He has been a signatory to the March 2023 Future of Life Institute open letter calling for a six-month pause on training runs more powerful than GPT-4 and the May 2023 Center for AI Safety "Statement on AI Risk." He spoke at the inaugural UK AI Safety Summit at Bletchley Park in November 2023 and has testified before the UK House of Lords and the Canadian House of Commons on the risks of advanced AI.[9][11]
Conjecture's public stance has consistently treated advanced AI as a potential extinction-level threat. The company argued that current scaling-led approaches to artificial general intelligence produce systems whose internal computations are opaque, whose generalization properties are poorly understood, and whose capabilities are growing faster than alignment science. In Conjecture's framing, this combination constitutes an unmanaged civilizational risk.[2][12]
The company positioned itself within what is sometimes called the "doomer" or high x-risk camp of AI safety, in contrast to more measured or capabilities-optimistic positions held by other labs. Leahy has been described in popular coverage as "the world's second-most famous AI doomer" and engaged in widely circulated debates with figures such as the e/acc movement's Beff Jezos, arguing that recursive self-improvement and the absence of any remaining "secret sauce" beyond scale make a near-term transition to superintelligence plausible. These exchanges, sometimes referred to in informal discourse as the "Foom-Doom debate," helped shape the broader public conversation about AI catastrophe scenarios.[11][12]
In a March 2023 interview with Sifted, Leahy summarized the danger bluntly: "If they just get more and more powerful, without getting more controllable, we are super, super fucked... And by 'we' I mean all of us." In the same interview he argued that the pace of frontier releases left no time to understand the models, stating that "GPT-4 should never have been released when it was."[12]
Conjecture's preferred policy framing, articulated repeatedly by Leahy and Miotti, called for: a cap on training compute at around 10^24 floating-point operations (roughly the order of ChatGPT-era models), strict liability for AI-induced harms applying both to developers and users, and the creation of governmental infrastructure to halt or "switch off" training runs and deployments deemed dangerous. Leahy presented this framework to the UK House of Lords in 2023.[11]
In its first year (2022), Conjecture's research portfolio covered several areas of applied alignment. Published or publicly discussed work included: scaling experiments on language models, mechanistic interpretability investigations, conceptual alignment research, and theoretical pieces such as the "simulator theory" framing of large language models and the "polytope lens" analysis of neural network internals.[2][7]
The company also operated Refine, a London-based incubator that ran in 2022 under Adam Shimi, designed as a three-month fully-paid fellowship to help aspiring independent researchers find, formulate, and seek funding for new conceptual alignment research bets. Refine was one of several attempts in the alignment field to broaden the supply of researchers pursuing genuinely novel theoretical agendas. Shimi later reflected publicly that Refine had not achieved its original goals and the program was wound down.[13]
During this initial phase Conjecture also developed engineering capacity for working with frontier models, including custom training infrastructure. In April 2023 the company filed a US trademark for "Verbalize," an automatic transcription product released in early 2023 that was positioned as a commercial offering. Critics noted that Verbalize did not appear to outperform existing open-source baselines such as Whisper, but Conjecture framed product work as a way to generate revenue and engineering know-how that could be reinvested in safety research.[14][7]
By late 2022, Conjecture's internal review had concluded that mainstream alignment approaches were unlikely to scale to superhuman systems. In response, the company narrowed its agenda around a new research direction, formally introduced on 25 February 2023 as Cognitive Emulation (CoEm).[15]
The CoEm proposal, posted by Leahy and collaborators as "Cognitive Emulation: A Naive AI Safety Proposal" on the AI Alignment Forum, advocated building "predictably boundable systems, not directly aligned AGIs."[15] The intuition behind CoEm is that, rather than attempting to align an opaque end-to-end superintelligent system, researchers should construct AI architectures whose components emulate human-style reasoning at human capability levels, with capabilities deliberately bounded and failure modes constrained to those that humans can recognize and respond to. As Conjecture put it, the aim is to build systems that are "good at chess for the same reasons humans are good at chess."[15][16]
Conjecture summarized the approach as creating "a system that reasons like a human and fails like a human." The architecture envisioned modular components, each interpretable and of limited intellect, that compose into a system whose overall behavior remains within an envelope of human-understandable cognition. The proposal emphasizes "logical" rather than "physical" emulation: developers care about whether functional outcomes match human reasoning patterns, not about replicating biological mechanisms. CoEms deliberately minimize what the authors call "Magic," defined as blackbox or not-understood computation; while CoEm components may still contain some opaque sub-modules, the architecture tracks and constrains where such "Magic" can appear.[15]
CoEm attracted significant discussion in the alignment community. Sympathetic readers viewed it as a serious attempt to articulate what controllable AI would actually look like in practice. Critics, including the authors of an anonymous EA Forum critique published in mid-2023, argued that the proposal remained underspecified and that the empirical work supporting it was thin.[7] Conjecture continued developing CoEm through 2023 and 2024 and described 2024 as its most scientifically productive year, publishing a "Roadmap for Cognitive Software and A Humanist Future of AI" in December 2024 and demonstrating internal tooling described as the "Tactics" framework.[4][2]
On 31 October 2024, Conjecture published "The Compendium: A Full Argument About Extinction Risk from AGI," an extended essay co-authored by Connor Leahy, Gabriel Alfour, Chris Scammell, Andrea Miotti, and Adam Shimi. The Compendium was simultaneously released as a dedicated website (thecompendium.ai), a downloadable PDF, and an accompanying podcast series, with cross-posts on LessWrong, the AI Alignment Forum, and the EA Forum.[1][17]
The Compendium aimed to present, in one accessible document, the authors' full argument for why humanity faces extinction risk from the race to AGI. It was written for non-technical readers and structured to walk through: the current AI landscape and competitive dynamics; the conceptual basis for expecting AGI to be created within the foreseeable future; the argument that sufficiently capable AI systems would not remain under human control; and policy and civic responses the authors considered necessary.[1][17]
A central rhetorical move in the Compendium is the concept of "intelligence domination": the claim that the relationship between a much more intelligent agent and a less intelligent one is, by historical analogy, naturally one of total domination. The essay argues that a superintelligent AI would relate to humanity as humans relate to less intelligent species, and that the existential risk arises not because such systems would be malicious but because humanity would become powerless to influence their decisions.[1] The authors maintain a public changelog and have described the document as a "living" text, with version 1.3.1 released on 9 December 2024.[17]
The Compendium was widely read inside the AI safety community and represented an attempt to consolidate the more pessimistic strand of x-risk argumentation in a form that could be used in policy and media settings. It also served, in effect, as a unifying intellectual statement for the cluster of people associated with Conjecture and ControlAI.
Andrea Miotti joined Conjecture in its early period and served as Head of Strategy and Governance. In this role he led much of the company's external policy engagement, including written and oral testimony to legislators in the UK, EU, and US. Miotti subsequently founded ControlAI, a nonprofit dedicated to mitigating risks from advanced AI systems, where he serves as Founder and Executive Director (often referred to as CEO).[18][19]
ControlAI is institutionally distinct from Conjecture but heavily overlapping in personnel and intellectual orientation. Gabriel Alfour, Conjecture's co-founder, helped Miotti establish ControlAI, and the two organizations have collaborated closely on public-facing projects. The flagship ControlAI policy document, "A Narrow Path," released by Miotti in October 2024, lays out a three-phase plan (Safety, Stability, Flourishing) for international governance of advanced AI, including compute thresholds, licensing regimes for frontier systems, and institutional mechanisms to enforce limits on dangerous capabilities. "A Narrow Path" closely mirrors the policy framework Leahy and Miotti had been advocating publicly since 2023.[20]
In early 2025 ControlAI shifted strategy toward what it called direct, simple, and honest civic action, focusing on building grassroots and parliamentary support for binding regulation of superintelligence. Conjecture's leadership credited this pivot with substantial unexpected success, including endorsements from over a hundred lawmakers and parliamentary testimony in multiple jurisdictions.[4][21] When Leahy announced Conjecture's wind-down in March 2026, his stated next role was as US Director of ControlAI, based in Washington, DC, completing the institutional transition from Conjecture-as-research-lab toward ControlAI-as-policy-organization.[4]
Conjecture and its leadership cultivated an unusually direct public communication style for an AI lab. Leahy, in particular, became a prolific podcast guest, debate participant, and conference speaker, often appearing on AI-focused programs such as Machine Learning Street Talk, Bankless, and The Inside View, as well as in mainstream coverage by Bloomberg, the Financial Times, Time, and Sifted.[22]
The company favored vivid, often confrontational rhetoric. Leahy frequently described the trajectory of frontier AI development in stark moral terms, repeatedly told journalists that humanity was "super fucked" if current trends continued, and rejected the framing of AI risk as merely a technical or economic question. This approach attracted both supporters, who valued its clarity, and critics, who argued that it risked alienating policymakers and undermining careful technical discourse.[22][12]
In late 2023 and 2024 Conjecture's communications increasingly emphasized policy advocacy: speaking at the UK AI Safety Summit at Bletchley Park, addressing a House of Lords event in September 2023, testifying before the Canadian House of Commons, and submitting written evidence to UK parliamentary committees jointly with collaborators such as Steven Adler.[11][21]
Conjecture raised approximately $10 million in 2022, primarily through commercial equity investment rather than philanthropic grants. Nat Friedman (former CEO of GitHub) was reported to be the company's first investor, approaching Leahy about starting a company. Other reported investors included Daniel Gross, Patrick and John Collison (co-founders of Stripe), Andrej Karpathy, and Arthur Breitman, among other technology entrepreneurs and angel investors.[7][23] Some early reporting also linked Sam Bankman-Fried's Alameda-era network to Conjecture's funding base, though the practical significance of that connection was limited by the subsequent collapse of FTX.[7]
This funding composition was unusual within the AI safety ecosystem. Where peers such as Redwood Research or FAR.AI relied predominantly on grants from sources such as Open Philanthropy and the Survival and Flourishing Fund, Conjecture's reliance on venture capital created different incentives. The company argued that this structure allowed it to act faster and on a larger scale than grant-dependent labs, but critics raised concerns about the alignment between investor expectations and a purely safety-oriented mission.[7]
Conjecture supplemented investment revenue with commercial product work, including the Verbalize transcription tool, and later with a roadmap for "Cognitive Software" products derived from CoEm research. By the company's own 2026 retrospective, the unit economics of frontier-scale research proved unsustainable for an organization of Conjecture's size relative to far-better-capitalized competitors.[4]
Conjecture occupied a polarizing position within the AI safety research community. Supporters credited it with producing some of the most accessible and forceful public articulations of x-risk concerns, with introducing the CoEm framing as a serious alternative to scaling-led alignment, and with bridging technical research and political advocacy at a moment when AI governance was rapidly developing.[22]
Critics, most prominently the anonymous "Omega" authors of a critique posted to the EA Forum and LessWrong on 12 June 2023 (later expanded and updated), raised concerns on several axes. The critique argued that Conjecture's publicly available research, including the simulator theory and polytope lens papers, fell below the standard expected of an organization of its scale and funding level; that the company had scaled headcount from around four to over twenty staff without commensurate research output; that there was tension between for-profit incentives and safety mission; that internal infohazard policies concentrated power in the CEO; and that Leahy's external communications had at times misrepresented Conjecture's focus to different audiences. The critique recommended against further funding until governance issues were addressed.[7]
Conjecture's leadership and several community members responded publicly, defending the company's approach and contesting the methodology of the critique. The episode nonetheless became a reference point in subsequent discussions of how AI safety organizations should be evaluated, alongside parallel critiques of other labs in the "Omega" series.[7]
Beyond the specific 2023 critique, Conjecture's broader rhetorical style and policy proposals attracted disagreement from both more measured AI safety researchers, who considered its public messaging counterproductive, and from accelerationist commentators, who rejected the underlying x-risk framing. The company's debates with these groups, including high-profile podcast exchanges, were widely circulated.[11]
Conjecture described 2024 as its most scientifically productive year, with continued CoEm research, the December 2024 publication of "A Roadmap for Cognitive Software and A Humanist Future of AI," and the October 2024 release of The Compendium.[2][17] In parallel, the company increased its policy engagement, with senior staff testifying before legislators in multiple jurisdictions and collaborating closely with the newly active ControlAI.[21]
According to the company's own retrospective written by Leahy in March 2026, Conjecture's 2025 did not unfold as planned. The economic conditions for independent frontier alignment research had become severe: the cost of competitive training runs continued to escalate while access to the necessary capital concentrated among a handful of well-funded competitors. Conjecture attempted to pivot toward commercial products derived from its CoEm and Cognitive Software research, but faced strong competition from larger and better-funded firms in adjacent markets. A final intensive product development sprint in December 2025 reportedly fell short of internal benchmarks.[4]
In parallel, ControlAI's early-2025 strategic pivot toward direct civic and parliamentary engagement produced what Leahy described as unexpected success, including support from over a hundred lawmakers and an invitation for him to testify before the Canadian House of Commons on the risks of superintelligence.[4][21] By early 2026 Conjecture's leadership had concluded that the marginal impact of continuing the lab was lower than that of dedicating its principal staff to ControlAI's policy work.
In March 2026, Leahy publicly announced that the Conjecture chapter was drawing to a close and that he would join ControlAI as its US Director, relocating to Washington, DC. Other former Conjecture staff continued in related roles, including Chris Scammell, who left COO duties at Conjecture to pursue a new project called the Buddhism and AI Initiative.[4][8] As of mid-2026, Conjecture's website continued to host its published research, and the broader intellectual program around Cognitive Emulation, The Compendium, and "A Narrow Path" remained influential in AI safety policy discussions, even as the original organization itself wound down.[2][17][20]
Conjecture's legacy in the AI safety field is therefore double-edged: as a research lab it produced a distinctive alignment agenda (CoEm) and a high-visibility argumentative framework (The Compendium), but as an institution it ultimately concluded that the marginal returns to additional technical research were lower than the returns to civic mobilization, and dissolved into the policy organization that several of its alumni had built. This trajectory, from technical alignment startup toward policy advocacy nonprofit, mirrors a broader 2024-2026 shift in parts of the AI safety community toward direct engagement with legislatures and regulators, including institutions such as the UK's Advanced Research and Invention Agency (ARIA) and parliamentary bodies in the US, UK, EU, and Canada.[21]
Imagine a small London company started by people who were really good at building AI but became convinced that AI was getting smart too fast and could one day be impossible for humans to control. Conjecture tried two things at once. First, it tried to invent a safer kind of AI built out of small, understandable pieces, so that the AI would "think like a person and make mistakes like a person" instead of being a mysterious black box. Second, it loudly warned governments and the public that super-smart AI could be dangerous, even writing a big online book called The Compendium to explain why. In the end, the company found it too expensive to keep competing on research, so in 2026 its founder shut Conjecture down and moved to a nonprofit, ControlAI, to focus full-time on getting laws passed about AI.