# AI Alignment Forum

> Source: https://aiwiki.ai/wiki/alignment_forum
> Updated: 2026-11-18
> Fact-checked: 2026-11-18
> Categories: AI Research, AI Safety
> License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) - attribute to "AI Wiki (aiwiki.ai)"
> Cite as: AI Wiki. "AI Alignment Forum." aiwiki.ai, 18 Nov 2026. https://aiwiki.ai/wiki/alignment_forum
> From AI Wiki (https://aiwiki.ai), the free encyclopedia of artificial intelligence. Reuse freely with attribution.

The **AI Alignment Forum** (alignmentforum.org, often shortened to AF) is an online forum for technical [AI alignment](https://aiwiki.ai/wiki/ai_alignment) research and discussion, built and run by the team behind [LessWrong](https://aiwiki.ai/wiki/lesswrong). It opened as a closed beta in July 2018 and launched publicly on October 29, 2018, replacing the Intelligent Agent Foundations Forum, which the [Machine Intelligence Research Institute](https://aiwiki.ai/wiki/miri) (MIRI) had launched in 2015.[1][2][3][4] Unlike LessWrong, where anyone can post, the Alignment Forum restricts direct posting and commenting to a small, selective membership, while every post and comment on it is automatically cross-posted to LessWrong, where anyone with an account can join the discussion.[1][5]

The forum's founders described its purpose as building "an online-hub for AI Alignment research," and chose a forum over a conference or journal on the grounds that it allowed faster and more thorough peer feedback.[1] It has since become a major venue for shorter, informal alignment writing. A 2022 study of the alignment literature collected 2,138 Alignment Forum articles alongside 707 arXiv manuscripts and found that more theoretical research tended to appear on the forum while more applied work tended to appear on [arXiv](https://aiwiki.ai/wiki/arxiv).[6] The forum is operated by Lightcone Infrastructure, the organisation that runs LessWrong.[8]

## History

### Predecessor: the Intelligent Agent Foundations Forum

On March 18, 2015, MIRI announced the Intelligent Agent Foundations Forum at agentfoundations.org, a forum "devoted to technical discussion of the research problems outlined in MIRI's technical agenda overview." It had already been active privately for several months before the public announcement.[4] By 2018 the LessWrong team wrote that agentfoundations.org had served as a venue for highly technical AI safety discussion, but that its design had become hard to maintain and hard for new researchers to join, and that the team wanted a venue that covered more than one research paradigm.[3]

### Beta (July to October 2018)

The LessWrong and AgentFoundations teams decided to build the new forum on the LessWrong codebase.[3] On July 10, 2018, Raymond Arnold (posting as Raemon) announced a closed beta of AlignmentForum.org on LessWrong; the October launch posts later referred to the period before launch as an open beta.[2][3] The post said the beta was meant to "iron out bugs and (more importantly) get feedback from researchers on whether the overall approach makes sense," and that the alignment community "has at least some needs that are best met with a semi-private forum."[3] During the beta a small number of admins invited members, who received posting and commenting rights; AF posts were shared to LessWrong, where anyone could comment, while only members could comment on the Alignment Forum itself.[3] A survey sent to everyone invited to the beta received 34 responses.[1]

### Launch (October 2018)

The forum launched on October 29, 2018. MIRI published an announcement by Oliver Habryka, the lead developer of LessWrong, describing the site as "a new hub for technical AI alignment research and discussion" and saying it would replace the Intelligent Agent Foundations Forum, which was being shut down and whose content was imported into the new site with MIRI's help.[2] An FAQ posted the same day by the development team, named as Oliver Habryka, Ben Pace, Raymond Arnold and Jim Babcock, said the forum had about 50 to 100 members drawn from researchers at the [Future of Humanity Institute](https://aiwiki.ai/wiki/future_of_humanity_institute) (FHI), the Center for Human-Compatible AI (CHAI), DeepMind, [OpenAI](https://aiwiki.ai/wiki/openai), MIRI, [Open Philanthropy](https://aiwiki.ai/wiki/open_philanthropy) and their collaborators, and that membership "will grow rarely and slowly."[1] The team said it planned to add one moderator with the power to invite new members from each of CHAI, DeepMind, OpenAI and MIRI.[1]

To seed the site, the launch posts announced three sequences that would run at about one post per day over the following weeks:[1][2]

| Sequence | Authors | Affiliation at launch |
|---|---|---|
| Embedded Agency | Scott Garrabrant and Abram Demski | MIRI |
| Iterated Amplification | [Paul Christiano](https://aiwiki.ai/wiki/paul_christiano) | OpenAI |
| Value Learning | Rohin Shah | CHAI |

The first Embedded Agency post, "Embedded Agents," by Abram Demski and Scott Garrabrant, appeared on launch day.[2][12]

### Changes in 2021

In June 2021 Peter Hase, then a PhD student working on machine learning interpretability, published a complaint on LessWrong titled "The Alignment Forum should have more transparent membership standards." He described applying for membership several times without a reply, being told that the main route to membership was to post on LessWrong and wait for moderators to promote the work, and being unable to answer comments on an AF post he had co-written because he was not a member.[9]

On August 24, 2021, close to the forum's third anniversary, the team published a new "Welcome & FAQ!" post. It said that several hundred researchers had by then contributed about two thousand posts and nine thousand comments to the forum. It said that non-members could now submit content directly on the Alignment Forum, where a decision to accept or reject it would be made within three days and any rejection would come with at least a one-sentence explanation, and that submissions would appear on LessWrong immediately in the meantime.[5] The same FAQ described oversight of major decisions by "an independent board composed of representatives of major Alignment research orgs (and independent members too)."[5] Its recommended reading for newcomers added Richard Ngo's AGI Safety from First Principles to the three launch sequences, and listed the [Alignment Research Center](https://aiwiki.ai/wiki/alignment_research_center) (ARC) among the organisations supporting research on the forum.[5]

## How the forum works

The Alignment Forum and LessWrong share one account system, and new posts and comments on the forum are copied to LessWrong automatically.[1][2] The main rules, as set out in the 2018 and 2021 FAQs, are:[1][5]

| Feature | How it works |
|---|---|
| Members | Can post and comment directly, and can promote LessWrong posts and comments to the forum |
| Non-members | Can read everything, comment on the LessWrong copy of any AF post, and (since August 2021) submit posts for review |
| Promotion | Members or admins move strong LessWrong content onto the forum; authors keep full ownership of it |
| Karma | AF karma is tracked separately from LessWrong karma; LessWrong shows both scores |
| Membership | Offered by the LessWrong and AF admins, who watch both sites for strong alignment contributors |
| Governance | Major decisions about the forum overseen by an independent board |

The 2021 FAQ said that working on alignment professionally was not by itself enough for membership, which was likely to be offered only after several existing members were excited by a person's work, and it recommended that would-be members write alignment content on LessWrong that gets promoted.[5] Habryka's 2018 announcement said that users whose content was frequently promoted and who earned strong karma from members would be recommended to the moderators as candidates for membership.[2] Under the beta rules, a LessWrong user whose comment an admin promoted to the forum gained voting privileges there and could start to accrue AF karma.[3]

## Role in alignment research

Habryka later wrote that the forum's goal had been to move the alignment field from relying mainly on "direct personal conversations with a few core researchers (at the time focused around MIRI and Paul Christiano)" to one whose core ideas could be learned online, and quoted Rohin Shah, then at DeepMind, as writing in 2020 that "the AI Alignment Forum improved our pedagogic materials from 0.1 to 3 out of 10."[8] Both statements appear in a fundraising post written by the forum's operators.

Independent measurement of the forum's place in the field comes from a 2022 arXiv paper by Jan H. Kirchner, Logan Smith, Jacques Thibodeau, Kyle McDonell and Laria Reynolds, which assembled a dataset of alignment research. The authors chose the Alignment Forum and arXiv as their two main sources because together they held a large share of published alignment research in a consistent form. Counting articles on both platforms, they found that output rose from fewer than 20 articles a year in 2017 to more than 400 a year by 2022.[6] They also found that AF articles were usually written by a single author or a team of fewer than five, unlike the long-tailed author counts on arXiv, and attributed this partly to the late introduction of a multiple-authors feature on the forum, while suggesting it might also reflect the forum's focus on speed of communication.[6]

The forum also served as a distribution channel for other alignment writing. Rohin Shah wrote in April 2019 that his Alignment Newsletter, which had 889 subscribers after its first year, was cross-posted to LessWrong, the Alignment Forum and Twitter, and that the amount of writing he wanted to cover had grown considerably "with the launch of the Alignment Forum."[7]

### Notable posts

Many documents that later became reference points in AI safety discussion were posted on the forum. The table lists a selection, with dates as recorded by the site.

| Date | Post | Author(s) | Notes |
|---|---|---|---|
| 2018-10-29 | "Embedded Agents" | Abram Demski, Scott Garrabrant | Opened the Embedded Agency sequence on launch day[12] |
| 2019-03-17 | "What failure looks like" | Paul Christiano | Two failure stories, "going out with a whimper" and "going out with a bang"[13] |
| 2019-05-31 | "Risks from Learned Optimization: Introduction" | [Evan Hubinger](https://aiwiki.ai/wiki/evan_hubinger) and four co-authors | Opened the sequence on [mesa-optimization](https://aiwiki.ai/wiki/mesa_optimization)[14] |
| 2020-07-27 | "Are we in an AI overhang?" | Andy Jones | Argued that [GPT-3](https://aiwiki.ai/wiki/gpt-3) left an [overhang](https://aiwiki.ai/wiki/capability_overhang)[15] |
| 2021-12-14 | "ARC's first technical report: Eliciting Latent Knowledge" | Paul Christiano, Mark Xu, Ajeya Cotra | ARC's first technical report, on [eliciting latent knowledge](https://aiwiki.ai/wiki/eliciting_latent_knowledge)[16] |
| 2022-06-05 | "AGI Ruin: A List of Lethalities" | [Eliezer Yudkowsky](https://aiwiki.ai/wiki/eliezer_yudkowsky) | Yudkowsky's list of reasons for expecting AGI to be lethal[17] |
| 2022-06-19 | "Where I agree and disagree with Eliezer" | Paul Christiano | Partly a response to "AGI Ruin"[18] |
| 2022-09-02 | "Simulators" | janus | Frame for self-supervised models, written at Conjecture[19] |
| 2023-02-05 | "SolidGoldMagikarp (plus, prompt generation)" | Jessica Rumbelow, mwatkins | Anomalous tokens in GPT-2 and GPT-3, work done at SERI-MATS[20] |
| 2024-12-18 | "Alignment Faking in Large Language Models" | Ryan Greenblatt, Evan Hubinger and others | Anthropic and Redwood Research paper on [alignment faking](https://aiwiki.ai/wiki/alignment_faking) in Claude 3 Opus[21] |
| 2025-04-03 | "AI 2027: What Superintelligence Looks Like" | [Daniel Kokotajlo](https://aiwiki.ai/wiki/daniel_kokotajlo) and co-authors | Scenario forecast [AI 2027](https://aiwiki.ai/wiki/ai_2027)[22] |

## Criticism and debate

Discussion of the forum's limits has mostly taken place on the forum and on LessWrong itself. Hase's 2021 post argued that an opaque membership process was a bad signal for the AI safety community, after he had been unable to join despite working in machine learning interpretability and co-writing a review of interpretability research that was posted on the forum.[9]

In January 2021 the researcher Adam Shimi argued in "The Case for a Journal of AI Alignment" that research confined to arXiv or the Alignment Forum "can be really hard to get any sort of deep feedback on," and that without a central peer-reviewed venue most people followed specific researchers and the Alignment Forum, "which is not the best for openness to new ideas." He proposed a dedicated journal with an editorial board modelled partly on the forum's own arrangement of one editor from each major lab.[10] In March 2021 Shimi, Joe Collman and Jérémy Perret started a sequence titled "Reviews for the Alignment Forum," in which they reviewed AF posts "to get a better idea of what a peer-review for AI Alignment would look like."[11]

## Operation and funding

The forum is maintained and run by the LessWrong team.[5] In its fundraising post for 2026, published on December 13, 2025, Lightcone described itself as "the organization behind LessWrong, Lighthaven, the AI 2027 website, the AI Alignment Forum, and many other things" and said it needed about $2 million to get through the following year.[8] The same post said that Open Philanthropy (by then renamed Coefficient Giving) appeared to have stopped recommending grants to Lightcone, and that the Survival and Flourishing Fund was the only large foundation funding its work, and only partially.[8] The post also quoted Eli Rose, a grantmaker at Open Philanthropy who had worked on several of its earlier grants to Lightcone, as saying, in a comment on the previous year's fundraiser, that LessWrong and the Alignment Forum had been "strongly positive" for building the human capital and knowledge base directed at AI safety.[8]

## See also

- [LessWrong](https://aiwiki.ai/wiki/lesswrong)
- [AI alignment](https://aiwiki.ai/wiki/ai_alignment)
- [AI safety](https://aiwiki.ai/wiki/ai_safety)
- [Machine Intelligence Research Institute](https://aiwiki.ai/wiki/miri)
- [Effective altruism](https://aiwiki.ai/wiki/effective_altruism)

## References

1. Habryka, O., Pace, B., Arnold, R. (Raemon) and Babcock, J. (jimrandomh). "Introducing the AI Alignment Forum (FAQ)". LessWrong, 2018-10-29. https://www.lesswrong.com/posts/FoiiRDC3EhjHx7ayY/introducing-the-ai-alignment-forum-faq
2. Habryka, O. (guest post). "Announcing the new AI Alignment Forum". Machine Intelligence Research Institute, 2018-10-29. https://intelligence.org/2018/10/29/announcing-the-ai-alignment-forum/
3. Raemon (Raymond Arnold). "Announcing AlignmentForum.org Beta". LessWrong, 2018-07-10. https://www.lesswrong.com/posts/JiMAMNAb55Qq24nES/announcing-alignmentforum-org-beta
4. Muehlhauser, L. "Introducing the Intelligent Agent Foundations Forum". Machine Intelligence Research Institute, 2015-03-18. https://intelligence.org/2015/03/18/introducing-intelligent-agent-foundations-forum/
5. Ruby and habryka. "Welcome & FAQ!". AI Alignment Forum, 2021-08-24. https://www.alignmentforum.org/posts/Yp2vYb4zHXEeoTkJc/welcome-and-faq
6. Kirchner, J. H., Smith, L., Thibodeau, J., McDonell, K. and Reynolds, L. "Researching Alignment Research: Unsupervised Analysis". arXiv:2206.02841, 2022-06-06. https://arxiv.org/abs/2206.02841
7. Shah, R. "Alignment Newsletter One Year Retrospective". AI Alignment Forum, 2019-04-10. https://www.alignmentforum.org/posts/3onCb5ph3ywLQZMX2/alignment-newsletter-one-year-retrospective
8. habryka (Oliver Habryka). "Toss a bitcoin to your Lightcone - LW + Lighthaven's 2026 fundraiser". LessWrong, 2025-12-13. https://www.lesswrong.com/posts/eKGdCNdKjvTBG9i6y/toss-a-bitcoin-to-your-lightcone-lw-lighthaven-s-2026
9. Hase, P. "The Alignment Forum should have more transparent membership standards". LessWrong, 2021-06-04. https://www.lesswrong.com/posts/dsr9vvcLjyLZGuBnN/the-alignment-forum-should-have-more-transparent-membership
10. Shimi, A. (adamShimi). "The Case for a Journal of AI Alignment". AI Alignment Forum, 2021-01-09. https://www.alignmentforum.org/posts/hNNM6gP5yZcHffmpD/the-case-for-a-journal-of-ai-alignment
11. Shimi, A., Collman, J. and Perret, J. "Reviews for the Alignment Forum" (sequence). AI Alignment Forum, created 2021-03-28. https://www.alignmentforum.org/s/pvXAS868E2BZEc2Fu
12. Demski, A. (abramdemski) and Garrabrant, S. "Embedded Agents". AI Alignment Forum, 2018-10-29. https://www.alignmentforum.org/posts/p7x32SEt43ZMC9r7r/embedded-agents
13. Christiano, P. (paulfchristiano). "What failure looks like". AI Alignment Forum, 2019-03-17. https://www.alignmentforum.org/posts/HBxe6wdjxK239zajf/what-failure-looks-like
14. Hubinger, E. (evhub), van Merwijk, C., Mikulik, V., Skalse, J. and Garrabrant, S. "Risks from Learned Optimization: Introduction". AI Alignment Forum, 2019-05-31. https://www.alignmentforum.org/posts/FkgsxrGf3QxhfLWHG/risks-from-learned-optimization-introduction
15. Jones, A. "Are we in an AI overhang?". AI Alignment Forum, 2020-07-27. https://www.alignmentforum.org/posts/N6vZEnCn6A95Xn39p/are-we-in-an-ai-overhang
16. Christiano, P. (paulfchristiano), Xu, M. and Cotra, A. "ARC's first technical report: Eliciting Latent Knowledge". AI Alignment Forum, 2021-12-14. https://www.alignmentforum.org/posts/qHCDysDnvhteW7kRd/arc-s-first-technical-report-eliciting-latent-knowledge
17. Yudkowsky, E. "AGI Ruin: A List of Lethalities". AI Alignment Forum, 2022-06-05. https://www.alignmentforum.org/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a-list-of-lethalities
18. Christiano, P. (paulfchristiano). "Where I agree and disagree with Eliezer". AI Alignment Forum, 2022-06-19. https://www.alignmentforum.org/posts/CoZhXrhpQxpy9xw9y/where-i-agree-and-disagree-with-eliezer
19. janus. "Simulators". AI Alignment Forum, 2022-09-02. https://www.alignmentforum.org/posts/vJFdjigzmcXMhNTsx/simulators
20. Rumbelow, J. and mwatkins. "SolidGoldMagikarp (plus, prompt generation)". AI Alignment Forum, 2023-02-05. https://www.alignmentforum.org/posts/aPeJE8bSo6rAFoLqg/solidgoldmagikarp-plus-prompt-generation
21. Greenblatt, R. (ryan_greenblatt), Hubinger, E. (evhub), Denison, C., Wright, B., Roger, F., Monte M, Marks, S., Treutlein, J., Bowman, S. and Buck. "Alignment Faking in Large Language Models". AI Alignment Forum, 2024-12-18. https://www.alignmentforum.org/posts/njAZwT8nkHnjipJku/alignment-faking-in-large-language-models
22. Kokotajlo, D., Larsen, T., elifland, Alexander, S., Jonas V and romeo. "AI 2027: What Superintelligence Looks Like". AI Alignment Forum, 2025-04-03. https://www.alignmentforum.org/posts/TpSFoqoG2M5MAAesg/ai-2027-what-superintelligence-looks-like-1

