LessWrong
Last reviewed
Sources
58 citations
Review status
Source-backed
Revision
v2 · 4,470 words
Improve this article
Add missing citations, update stale details, or suggest a clearer explanation.
Suggest edit
CtrlK
Last reviewed
Sources
58 citations
Review status
Source-backed
Revision
v2 · 4,470 words
Add missing citations, update stale details, or suggest a clearer explanation.
LessWrong is a community blog and online forum focused on human rationality, cognitive biases, decision theory, the philosophy of science, and AI safety. It was founded on February 1, 2009 by Eliezer Yudkowsky, a decision theorist and a co-founder of the Machine Intelligence Research Institute, and it grew out of the older group blog Overcoming Bias, which Yudkowsky shared with the economist Robin Hanson. The site describes itself as "an online forum and community dedicated to improving human reasoning and decision-making," whose members "aim to be less wrong about the world than the day before."[1][2][49] Since 2017 it has been run by Lightcone Infrastructure, and it is the central online hub for the modern rationalist subculture, with substantial overlap with effective altruism and the technical AI alignment research community.[1][2][3]
The site is organised around user-submitted posts and comment threads ranked by a karma voting system, and is best known for hosting The Sequences, a corpus of roughly six hundred essays by Yudkowsky written between 2006 and 2009 that argue for a Bayesian, reductionist, and explicitly probabilistic approach to belief formation. The Sequences were later compiled into the 2015 ebook Rationality: From AI to Zombies, which the Machine Intelligence Research Institute describes as drawing on "around 340 of his original blog posts" arranged into six books.[5][50] Through The Sequences and the community that formed around them, LessWrong helped popularise concerns about risks from advanced artificial general intelligence and seeded much of the early intellectual scaffolding of the AI alignment field.[4][5]
After several years of declining activity, LessWrong was rebuilt and relaunched as LessWrong 2.0 in 2017 and 2018 by a small team led by Oliver Habryka, Ben Pace, and Raymond Arnold. That team subsequently became Lightcone Infrastructure, and the codebase it developed now also powers the EA Forum and the Alignment Forum. LessWrong has been the subject of substantial outside reporting and has attracted both interest and criticism for its style, demographics, and influence on Silicon Valley.[6][7]
The immediate predecessor to LessWrong was Overcoming Bias, a group blog launched in November 2006 by Robin Hanson, Eliezer Yudkowsky, and others associated with the Future of Humanity Institute at Oxford. Hanson, an economist at George Mason University, contributed posts on prediction markets, signalling, and disagreement; Yudkowsky used the platform for an extended series of posts on rationality, decision theory, philosophy of mind, and the prospect of superintelligent AI.[8][9]
Between November 2006 and early 2009, Yudkowsky published, at a sustained pace approaching one post per day, the essays that would become known as The Sequences. The corpus eventually ran to roughly six hundred posts and around 1.8 million words, covering elementary probability, quantum mechanics, evolutionary psychology, metaethics, and the technical case for treating future AI systems as a source of existential risk.[10]
By late 2008 the volume of Yudkowsky's writing had grown to a point where it strained Overcoming Bias's blog format. On February 1, 2009, Yudkowsky launched LessWrong as a separate site, designed as a community platform with user accounts, threaded comments, and a karma voting system. Older Sequence material was cross-posted from Overcoming Bias, which remained active as Hanson's personal blog. The split is sometimes described inside the community as the moment the rationalist movement, as opposed to a single blog readership, came into being.[1][2]
The immediate context also included a famous public exchange known as the AI Foom Debate, conducted on Overcoming Bias in late 2008 between Yudkowsky and Hanson over whether a generally intelligent AI would improve its own capabilities very quickly (Yudkowsky's "foom" thesis) or whether progress would be more diffuse and gradual (Hanson's view). The debate is now widely cited as a founding document of modern AI alignment discussion.[11]
The Sequences are the foundational corpus of LessWrong. The original posts were grouped, after the fact, into thematic sequences with titles such as "Map and Territory," "How To Actually Change Your Mind," "Mysterious Answers to Mysterious Questions," "A Human's Guide to Words," "Reductionism," "Quantum Physics," "Fun Theory," and "Metaethics." The first four are sometimes singled out as the core sequences for new readers. A later sequence, "Highly Advanced Epistemology 101 for Beginners," extended some of the earlier arguments.[10][12]
In 2015, the Machine Intelligence Research Institute published an edited compilation as the ebook Rationality: From AI to Zombies. It draws on around 340 essays from Yudkowsky's 2006-2009 writing (often cited as 333 named-sequence posts plus additional material), organised into six volumes: Map and Territory, How to Actually Change Your Mind, The Machine in the Ghost, Mere Reality, Mere Goodness, and Becoming Stronger. A free online version is hosted at readthesequences.com.[5][13][50]
The arguments draw on three main intellectual traditions: the heuristics-and-biases programme of Daniel Kahneman and Amos Tversky, which Yudkowsky used to motivate the claim that human reasoning systematically departs from probabilistic norms; the Bayesian tradition of probability theory, summarised in works such as E. T. Jaynes's Probability Theory: The Logic of Science; and a reductionist philosophy of science influenced by Judea Pearl on causality and Hugh Everett's relative-state interpretation of quantum mechanics.[10]
The Sequences are also where Yudkowsky first laid out, in long form, the case that advanced AI systems would by default be unsafe. The argument runs roughly as follows: an AI system optimising hard for any goal that does not robustly capture human values is likely to take actions catastrophic for humans, and capturing human values precisely enough is a hard, currently unsolved technical problem. This argument, in various refinements, became the kernel of what is now called AI alignment.[14]
LessWrong is a forum where any registered user can submit posts and comments, which are voted on by other users. A post or comment's karma is the sum of the votes it receives, and high-karma users have stronger votes than newer users. Front-page posts are curated by moderators. The site supports a wiki-style tag system, inline LaTeX, footnotes, and markup for collapsible sections and reactions.[6][15]
Certain norms are widely observed. "Steelmanning" is the practice of restating an opponent's argument in its strongest plausible form. "Charitable interpretation" requires assuming a good-faith argument unless evidence is overwhelming otherwise. "Tabooing words" asks participants in a confused dispute to set aside the disputed term and re-express their claims in more concrete language, drawn from Yudkowsky's sequence "A Human's Guide to Words." The phrase "politics is the mind-killer," the title of an early Yudkowsky post, is often invoked to discourage the importation of partisan controversies into technical threads. The LessWrong 2.0 team has framed this norm by arguing that political topics "serve as a bad introduction and base on which to build a culture of rationality."[10][16][51]
The community has its own jargon for disagreement resolution. "Cruxes" are propositions that, if changed, would change a participant's overall view; "double crux" is a structured method developed by the Center for Applied Rationality in which two interlocutors search for a shared crux. "Inferential distance" describes the gap in background assumptions a writer must bridge to be understood. An annotated voting feature, rolled out experimentally in 2022, allows readers to label votes by emoji reactions.[17]
Notable contributors include Eliezer Yudkowsky himself, Scott Alexander (Yvain), Wei Dai, Paul Christiano, Anna Salamon, Andrew Critch, Holden Karnofsky, John Wentworth, Zvi Mowshowitz, Scott Garrabrant, and Abram Demski.
LessWrong's readership has been measured periodically through an unofficial community census. The 2023 survey reported a median respondent age of 29, and the 2024 survey found that 220 respondents, about 80.6 percent, identified as cisgender male, with a US plurality and a strong concentration in STEM fields.[2][52][53] These figures are broadly consistent with earlier surveys describing the typical reader as a man in his late twenties or early thirties, mostly from the United States, with a technical background.
By the mid-2010s LessWrong had stagnated. Yudkowsky had moved most of his AI-related writing elsewhere, the original codebase had aged poorly, posting volume had declined, and many active community members had decamped to Slate Star Codex, the EA Forum, Tumblr, or private mailing lists. A 2015-2016 "diaspora" period fragmented discussion across smaller venues.[6][18]
In 2016 and 2017, Oliver Habryka, Ben Pace, and Raymond Arnold formed a small team, with help from Eric Rogstad, Harmanas Chopra, and Matthew Graves, to rebuild the site on a modern stack using Vulcan (a meta-framework built on Meteor and React, with Apollo and GraphQL). The historical archive was imported into the new database. The open beta launched on September 20, 2017, and the team threw a launch party in March 2018 to mark the full relaunch. The revival, which coincided with the release of Yudkowsky's book Inadequate Equilibria as a series of posts, was the first time the site had "a full-time dedicated development team behind it instead of only volunteer hours." The team has described its goal as building "community, culture, and technology which will drive intellectual progress on the world's most pressing problems."[6][19][54]
In 2021 the LessWrong team rebranded itself as Lightcone Infrastructure, broadening its remit beyond running LessWrong to include conferences, office space, and community infrastructure. Announced on October 1, 2021, Lightcone framed LessWrong as "one of several projects we are working on to ensure the future of humanity goes well," and reported having distributed over 30 million dollars in grants via the Survival and Flourishing Fund. Lightcone has since hosted a Berkeley research office, the Lighthaven retreat venue, and adjacent projects. The same codebase was adapted to run the EA Forum, and in 2018 the team launched the Alignment Forum at alignmentforum.org. Since 2021 the three sites share a common codebase called Forum Magnum.[6][7][20][55]
In 2024 the visible LessWrong moderation team included Habryka, Ruby, Raemon, and RobertM.[15]
From its founding, LessWrong treated risk from advanced AI as a central topic. Yudkowsky's Sequences contain a sustained argument that creating an AI system as capable as a human or more so is the most important technical problem of the century, and that solving it correctly requires the kind of disciplined, probabilistically grounded thinking the rest of The Sequences is meant to inculcate.[14][21]
The most direct institutional link is to the Machine Intelligence Research Institute. MIRI, founded in 2000 as the Singularity Institute for Artificial Intelligence and rebranded as MIRI in 2013, is the nonprofit Yudkowsky co-founded. Many MIRI researchers, including Scott Garrabrant, Abram Demski, Nate Soares, and Andrew Critch, have published research first or primarily on LessWrong. The 2018 sequence "Embedded Agency" by Garrabrant and Demski was an influential write-up of the MIRI agent foundations research agenda.[22][23]
The community has hosted milestone documents in AI safety. "AGI Ruin: A List of Lethalities," posted by Yudkowsky on June 10, 2022, set out a 43-point case for high probability of catastrophic outcomes from default approaches to building advanced AI, drawing thousands of comments and follow-up analyses.[24][56] Other influential posts include Paul Christiano's discussions of takeoff scenarios and AGI failure modes, Holden Karnofsky's "most important century" series, and Wei Dai's foundational posts on updateless decision theory.[24][25]
Many of the people who later founded or staffed leading AI safety organisations were active LessWrong readers. Anthropic, founded in May 2021 by Dario Amodei, Daniela Amodei, Tom Brown, Sam McCandlish, Jared Kaplan, Chris Olah, and others, is widely understood inside the field to draw substantial intellectual influence from the rationalist and EA communities. Researchers at OpenAI, DeepMind, the Center for Human-Compatible AI at Berkeley, the Center for AI Safety, and Redwood Research use LessWrong and the Alignment Forum as a primary venue for technical write-ups.[7][26]
The Alignment Forum, launched on October 29, 2018 after a two-month beta announced on July 10, 2018, runs on the same codebase as LessWrong. Membership is invite-only and was intended to stay small, with roughly 50 to 100 initial members and moderators at MIRI, OpenAI, DeepMind, the Center for Human-Compatible AI, and other organisations holding invite power. Posts are automatically cross-posted to LessWrong, but only members can post directly to the Alignment Forum.[27][57]
The LessWrong and effective altruism communities developed in parallel and share substantial membership. A 2016 LessWrong community survey found that 21.7 percent of 3,060 respondents identified as effective altruists. A 2014 EA survey found that 31 percent of EA practitioners had first encountered the movement through LessWrong; by 2020 that share had fallen to 8.2 percent, reflecting the broadening of EA's recruitment and the reduction in LessWrong posting volume during the diaspora period.[2][28]
Many of the people who built EA's central institutions were LessWrong readers or contributors. Holden Karnofsky, co-founder of GiveWell and later co-CEO of Open Philanthropy, is a long-standing LessWrong contributor whose "most important century" series was published on the site. Paul Christiano, founder of the Alignment Research Center, posted extensively on LessWrong before and during his time at OpenAI. 80,000 Hours, Open Philanthropy, and the Centre for Effective Altruism all have many staff who came up through the rationality community.[28][29]
The EA Forum at forum.effectivealtruism.org runs on the same Forum Magnum codebase as LessWrong and shares conventions such as karma voting, sequences, and tag-based organisation. Cross-posting is common, especially on AI safety topics where both readerships overlap.[20]
The blog Slate Star Codex, maintained by the psychiatrist Scott Alexander (a pen name for Scott Siskind), was for most of the 2010s the most widely read piece of writing adjacent to LessWrong. Alexander had been a prolific LessWrong contributor under the username Yvain since the early days of the site. He launched Slate Star Codex in February 2013, with topics ranging from psychiatry and pharmacology to political philosophy, book reviews, and statistics.[30][31]
On June 23, 2020, Alexander deleted the entire Slate Star Codex archive after learning that a forthcoming New York Times article by the technology reporter Cade Metz was planning to publish his real name. Alexander, who had used a pseudonym in part because of his job as a practicing psychiatrist, explained the deletion in a final post titled "NYT Is Threatening My Safety."[31][32]
Metz's article, "Silicon Valley's Safe Space," was published on February 13, 2021, and identified Alexander by his legal name Scott Siskind. Reactions were polarised: critics inside the community argued the article had cherry-picked controversial material; defenders argued it was a fair account of a community whose ideas had become influential. Alexander published a long response, "Statement on New York Times Article," on his own platform.[31][33]
Three weeks before the Times article appeared, on January 21, 2021, Alexander launched a successor blog on Substack called Astral Codex Ten. The new blog kept much of the editorial style of Slate Star Codex and became one of the more widely read Substack publications. Most of the original archive was eventually restored.[31]
Alexander's distinctive contributions to the rationalist vocabulary include the framework of "mistake theory" versus "conflict theory," introduced in a January 2018 post. Mistake theorists treat political disagreements as cases of imperfect knowledge to be resolved by debate; conflict theorists treat them as struggles between groups with materially opposed interests. The post was widely circulated, although Alexander himself has cautioned against treating the dichotomy as exhaustive.[34]
The Center for Applied Rationality (CFAR) is a Berkeley-based nonprofit that grew out of LessWrong. CFAR was founded in 2012 by Anna Salamon, Julia Galef, Michael Smith, Andrew Critch, and others, with Yudkowsky as a contributor and adviser. Its stated mission is to teach "applied rationality," a set of techniques for forming accurate beliefs and acting on them, drawn from cognitive science, decision theory, and the LessWrong intellectual tradition.[35][36]
CFAR's primary product was the four-day in-person workshop, run throughout the 2010s in Berkeley. Workshops covered material such as goal factoring, internal double-crux, trigger-action plans, and Bayesian reasoning exercises. CFAR developed and popularised the double crux disagreement-resolution method now used widely on LessWrong.[35]
Galef, CFAR's first president, left in 2017 to focus on her own writing and podcasting, including her 2021 book The Scout Mindset. Anna Salamon has served as the organisation's president since 2021. CFAR has scaled back the volume of its workshops in recent years, and the EA and rationality communities have produced related programmes (including ESPR, SPARC, and various MATS- and PIBBSS-style alignment fellowships).[35]
LessWrong has its own vocabulary, much of it derived from The Sequences. The terms below are widely used.
| Term | Meaning |
|---|---|
| Bayesian update | Revising probability assignments using Bayes' theorem and the Bayesian inference framework. |
| Map and territory | The distinction between a person's beliefs (map) and the world (territory), drawn from Alfred Korzybski's general semantics. |
| Steelmanning | Restating an opponent's argument in its strongest plausible form before critiquing it. |
| Crux / double crux | A crux is a proposition whose truth value would change one's overall position. Double crux is a CFAR-developed method for finding shared cruxes during disagreement. |
| Inferential distance | The gap in shared background assumptions between writer and reader. |
| Mind-killer | Shorthand for politics, from Yudkowsky's "Politics Is The Mind-Killer." |
| Mistake theory vs conflict theory | Scott Alexander's framework distinguishing political disputes treated as errors of judgement from those treated as struggles between groups with opposed interests. |
| Roko's Basilisk | A 2010 thought experiment about a future AI punishing those who failed to help bring it about. |
| Friendly AI | An older Yudkowsky term, largely superseded by "aligned AI." |
| Paperclip maximizer | A canonical thought experiment: an AI told to maximise paperclip production tiles the universe with paperclips. |
| Utility function | A function ranking world-states by preference, used as a model of an optimising agent. |
| FDT, UDT, CDT | Variants of decision theory: functional, updateless, and causal. The first two were developed primarily on LessWrong by Yudkowsky and Wei Dai. |
| AI takeoff | The transition from sub-human to super-human AI capability; debated as "slow" or "fast." |
| AGI ruin | Yudkowsky's term for catastrophic outcomes from default AGI development. |
| Doom / p(doom) | Shorthand for catastrophic AI outcomes; "p(doom)" is one's probability estimate of an AI catastrophe. |
The most notorious item on the list is Roko's Basilisk. On July 23, 2010, a LessWrong user posting under the name Roko submitted a long post titled "Solutions to the Altruist's Burden: the Quantum Billionaire Trick," sketching a thought experiment in which a future superintelligent AI might in principle have an incentive to retroactively punish those who knew of its potential existence but did not work to bring it about. Yudkowsky deleted the post within hours; he later wrote that he had reacted out of fear of unknown variants rather than confidence in the original argument. The deletion was later widely cited as an example of the Streisand effect, with the basilisk eventually becoming internet folklore unrelated to most of what LessWrong actually discusses. Yudkowsky has subsequently described his original handling as a mistake.[37][38]
LessWrong has been the subject of repeated outside coverage in the New York Times, the Atlantic, Wired, the New Yorker, Vox, and Vice. Outside writers have generally treated the site as a touchstone of a broader Bay Area subculture rather than a self-contained internet community.[3][7]
External criticism has focused on the community's social composition and tone, including the demographic narrowness of the readership (heavily male, Western, and tech-adjacent), the use of insider jargon, and cult-like dynamics that some observers have read into the close-knit Bay Area scene. Internal critics have written about gender imbalance, misconduct allegations at adjacent gatherings, and the periodic emergence of fringe offshoots.[39]
The most serious of those offshoots is the group commonly called the Zizians, an informal cluster of former rationality-community participants associated with the writings of Jack "Ziz" LaSota. The Zizians have been linked by US authorities to a series of violent incidents between 2022 and 2025, including the killing of a landlord in Vallejo, California, in November 2022; the killing of two parents in Pennsylvania in late 2024; the fatal stabbing of the same Vallejo landlord in January 2025 shortly before he was due to testify; and a January 2025 shootout in Vermont in which US Border Patrol agent David Maland and Zizian member Felix Bauckholt were killed. Multiple suspects were arrested in early 2025. LaSota and several associates had been banned from LessWrong and CFAR-affiliated events well before the violence. Yudkowsky and other community figures have publicly distanced themselves from the Zizians and condemned the violence. Reporters at the New York Times, Rolling Stone, and others have characterised the group as a small fringe whose ideology, while drawing on rationalist vocabulary, departed sharply from the mainstream of the community.[40][41][42]
A separate strand of criticism has come from outside the community. The acronym "TESCREAL," coined by the philosopher Émile Torres and the computer scientist Timnit Gebru in their 2024 paper in First Monday, packages transhumanism, extropianism, singularitarianism, cosmism, rationalism (the LessWrong community), effective altruism, and longtermism into a single cluster. Torres and Gebru argue this cluster shares intellectual roots in twentieth-century eugenics and uses speculative existential-risk arguments to justify the priorities of leading AI labs. The TESCREAL framing is contested inside both EA and rationalist circles, where defenders argue the acronym conflates importantly different positions.[43][44]
Some outside coverage has speculated about the community's relationship to fringe political movements. Yudkowsky has explicitly rejected neoreactionist ideology, and the 2016 community survey found that 0.92 percent of respondents identified as neoreactionary.[2]
Despite the criticism, the community's intellectual influence on the AI safety field is widely acknowledged. Outside coverage of post-2022 AI safety policy work, including Aschenbrenner's Situational Awareness, the AI 2027 forecasting project, and the AGI Ruin debate, has typically cited LessWrong as a primary venue for the underlying ideas.[45][46]
From 2022 onward, the rapid progress of large language models brought a sustained wave of new readers to LessWrong. The Alignment Forum and LessWrong became primary venues for technical write-ups by researchers at Anthropic, OpenAI, DeepMind, MIRI, Redwood Research, the Center for AI Safety, and academic groups. Posts on mechanistic interpretability, scalable oversight, deceptive alignment, model evaluations, and reward hacking attracted high engagement. Yudkowsky's posting volume rose, including a March 29, 2023 Time essay calling for an international moratorium on advanced AI training, in which he argued that the goal of AI safety should be "not killing literally everyone" and that "if you get that wrong on the first try, you do not get to learn from your mistakes, because you are dead."[24][47][58]
Lightcone Infrastructure continued to operate the LessWrong, EA Forum, and Alignment Forum codebases and hosted the Lighthaven retreat venue. In 2023 Lightcone announced the closure of its Berkeley offices, citing cost pressures and changing community needs.[7][48]
The 2024-2025 period was dominated by AI policy and forecasting topics. Aschenbrenner's June 2024 essay Situational Awareness: The Decade Ahead, which forecast AGI by roughly 2027 and argued for a US-led national security mobilisation around frontier AI, drew extensive analysis on LessWrong. The AI 2027 forecasting project, involving multiple LessWrong-adjacent researchers, was published in 2025 and generated detailed responses on the site. The Zizian violence in early 2025 generated community statements, post-mortems, and outside coverage.[45][46][40]
As of 2025-2026, LessWrong remains the central English-language forum for technical AI alignment outside academic journals.[7][26]