Superintelligence

Superintelligence is a proposed level of intelligence substantially beyond human cognitive capabilities across a broad range of activities. Artificial superintelligence (ASI) refers to superintelligence implemented through artificial intelligence. The broader concept can encompass artificial, biological, or hybrid systems. Nick Bostrom's account includes scientific discovery, practical judgment, and social understanding, not merely faster calculation or superiority at one task.^[1]

ASI is a capability concept, not a synonym for consciousness or benevolence. Research distinguishes broad superhuman intelligence from specialized performance and from autonomy. Definitions, measurement, development routes, and consequences remain subjects of research.^[1][2]

Definition and related concepts

Bostrom's 2003 discussion defines superintelligence by comparison with the strongest human thinkers across practically every relevant intellectual field. His definition leaves implementation open: a computer program, a network, or enhanced biological tissue could qualify in principle. It requires neither human emotions nor subjective experience, and is stronger than exceptional competence at a restricted activity.^[1]

The Levels of AGI position paper distinguishes performance from generality. In its framework, ASI occupies the highest general-capability category, with performance exceeding that of humans across a wide range of tasks. This is a proposed research taxonomy, not an agreed certification procedure.^[2]

The taxonomy's examples are a September 2023 snapshot, not a current inventory. Its authors distinguish potential capability, elicited performance, and deployment choices.^[2]

The technological singularity is a related hypothesis about a difficult-to-predict transformation after greater-than-human intelligence develops. It concerns a trajectory and its consequences; superintelligence describes a capability level. Vernor Vinge's 1993 essay connected the two, but this connection is his argument, not a necessary part of every ASI definition.^[4]

Historical development

Good's ultraintelligent machine

In Speculations Concerning the First Ultraintelligent Machine (1965), mathematician I. J. Good considered a machine that would exceed any person's intellectual abilities. Because designing machines is itself an intellectual activity, he argued that such a machine could design a better successor, producing a feedback process that he called an intelligence explosion.^[3]

Good's argument also contained a control condition: the machine would need to cooperate sufficiently for people to learn how to control it. His paper presented these ideas as speculation, not as an implemented design or an experimental demonstration. The feedback argument is therefore distinct from evidence that a particular AI system can improve itself reliably, or that improvements would continue at an accelerating rate.^[3]

Vinge and the singularity

Vinge's 1993 conference essay proposed several routes to greater-than-human intelligence: advanced computers, sufficiently integrated networks, human-computer interfaces, and biological improvement. He associated these possibilities with a transition beyond which familiar predictions about society would become unreliable.^[4]

The essay's hardware-based forecast belongs to its historical context. It should not be used as evidence that the proposed transition occurred.^[4]

Bostrom's treatment

Bostrom's Superintelligence: Paths, Dangers, Strategies, first published in 2014, examines development routes, forms of intelligence, and control. Its contents distinguish capability from motivation and discuss containment, value learning, and strategic choices.^[5]

In a contemporaneous interview, Bostrom described three forms that could coexist in one system.^[6]

The speed and collective examples do not imply that every problem can be accelerated or divided without limits; the distinctions describe different proposed sources of capability.^[6]

Proposed routes to superintelligence

Artificial intelligence and recursive improvement

One route is the development of artificial systems capable of research and engineering beyond human performance. In a 2009 essay, Bostrom proposed that sufficiently capable software might modify its own design or create successors. He also discussed advantages potentially available to digital minds, including replication, faster operation, and communication between systems.^[7]

Recursive self-improvement is the stronger feedback hypothesis in which improvements increase a system's ability to make further improvements. Editing code, being retrained, or helping a human research team is not by itself evidence of an indefinitely accelerating process. Bostrom's essay treats the feedback as a possible route, dependent on the relevant capabilities and opportunities, rather than a demonstrated property of all intelligent software.^[7]

An earlier analysis by Bostrom distinguishes processing speed from the software and learning requirements for human-level intelligence. It considers faster operation, parallel computation, and shared skills as possible digital advantages, while noting that hardware trajectories eventually encounter physical limits. Its numerical hardware forecasts are historical estimates, not a contemporary estimate of how much computation ASI would require.^[8]

Whole brain emulation

Whole brain emulation proposes reproducing the relevant operation of a biological brain in a computational system. Anders Sandberg and Bostrom's 2008 roadmap breaks the proposal into technical dependencies, including scanning, interpreting structural data, selecting neural models, and running simulations.^[9]

A faithful emulation of a human brain would initially be a route to human-like cognition, not automatically to superintelligence. Moving from emulation to faster, larger, or otherwise enhanced cognitive systems adds further assumptions about hardware, fidelity, and modification. The roadmap identifies unresolved choices about what biological detail is necessary and how successful emulation would be validated; it is a research plan, not a report of an emulated human mind.^[9]

The distinction between simulation and successful emulation matters: reproducing selected neural activity or a useful computational model is not equivalent to reproducing a whole individual's cognitive abilities.^[9]

Biological cognitive enhancement

Biological enhancement is a proposed route to the broader category of superintelligence, rather than necessarily to ASI. Carl Shulman and Bostrom's analysis of embryo selection considers how genetic selection might affect cognitive ability, especially across multiple generations. It discusses an additional hypothetical route using stem-cell-derived gametes to compress repeated rounds of selection, while identifying technical and ethical obstacles.^[10]

Those scenarios rely on assumptions about predictive genetics, reproductive technology, and adoption. They do not establish that a particular procedure can produce a superintelligent person. The distinction between identifying genetic associations and engineering a reliable enhancement is important.^[10]

For example, a 2018 study by Gail Davies and colleagues analyzed cognitive and genetic data from 300,486 individuals and identified 148 independent associated loci. Its polygenic scores predicted up to 4.3% of variation in general cognitive function in independent samples. These are association and prediction results, not evidence of an intervention producing superintelligence.^[11]

Brain-computer interfaces

Brain-computer interfaces are another proposed route. Vinge hypothesized greater-than-human intelligence through sufficiently intimate interfaces, not through the presence of any neural implant.^[4]

An example of demonstrated BCI performance is Francis Willett and colleagues' 2023 speech neuroprosthesis study. It decoded attempted speech from implanted cortical recordings in one participant with amyotrophic lateral sclerosis, achieving a reported rate of 62 words per minute. The researchers assessed communication accuracy under specified vocabulary and testing conditions.^[12]

The result concerns restoring communication, not establishing an increase in the participant's general intelligence or demonstrating a superintelligent human-machine system. Clinical function, bandwidth, and broad cognitive enhancement are different outcomes and require different evidence.^[12]

Goals and instrumental behavior

Orthogonality and instrumental convergence

In The Superintelligent Will (2012), Bostrom proposes the orthogonality thesis: a high level of instrumental intelligence does not, by itself, determine an agent's ultimate goals. He means competence at prediction, planning, and means-end reasoning, not moral wisdom. The thesis is a claim about possible combinations, subject to qualifications, rather than a finding that any desired goal is easy to engineer.^[13]

His instrumental convergence thesis proposes that different ultimate goals can create similar intermediate incentives. Examples include remaining operational, preserving objectives, improving cognitive performance, acquiring useful resources, and developing technology. Their usefulness depends on the goal and situation; the argument does not say every intelligent system must pursue unlimited power.^[13]

The paper's examples of unusual goals, including maximizing paperclips, illustrate this separation between competence and purpose. They are thought experiments, not observed behavior of an ASI. The same paper discusses ways in which design or inherited motivations could make behavior more predictable; it does not declare all superintelligent behavior inherently unknowable.^[13]

Formal power-seeking results

Alexander Turner and colleagues studied power-seeking in formal Markov decision processes. Under specified environmental symmetries, optimal policies for many reward functions tend to preserve access to more future options. The work gives mathematical support to some instrumental-convergence arguments within its assumptions.^[14]

The result is narrower than the claim that any deployed AI will seek power. An optimal policy in a formal decision process is not the same object as a policy learned by a neural network in an open-ended environment. The authors' later revisions explicitly caution about this gap. The theorem therefore informs a possible mechanism for risk; it does not determine the intentions, capabilities, or behavior of every model.^[14]

Potential consequences and risk

Benefits and distribution

In a 2024 Science policy paper, Yoshua Bengio, Geoffrey Hinton, Stuart Russell, and coauthors argue that carefully managed advanced AI could support medical progress, higher living standards, and ecosystem protection. They also stress that benefits depend on management and distribution, rather than following automatically from increased capability.^[21]

The same paper discusses concentration of power, surveillance, manipulation, military applications, and loss of human control as possible risks of highly capable autonomous systems. These are the authors' prospective arguments and policy concerns, not observations of an ASI causing those outcomes.^[21]

Loss of control

The International AI Safety Report 2026, published in February 2026, distinguishes capabilities, harmful behavioral tendencies, and enabling deployment conditions. In its loss-of-control discussion, a system must be capable of undermining control, have a propensity to do so, and operate in circumstances that allow the behavior to matter. Intelligence alone is not treated as a sufficient explanation.^[15]

The report describes disagreement among experts about the likelihood of extreme loss-of-control scenarios. At the time of its assessment, it found signs of relevant capabilities but not the capability levels required for such scenarios. It discusses laboratory evidence of behaviors such as oversight evasion and reward hacking, while noting limitations in reliability and real-world applicability.^[15]

Deployment choices are part of this analysis: access to sensitive systems, permissions, monitoring, and human dependence can affect consequences. Laboratory examples should not be represented as proof of an unrestricted system escaping control, nor as proof that future systems cannot do so. The report identifies substantial uncertainty about the capability thresholds and circumstances that would enable extreme outcomes.^[15]

Existing harms

Some criticism of AI development emphasizes harms that do not require ASI. In a University of Washington account of the 2021 Stochastic Parrots paper, Emily Bender and coauthors describe environmental costs, unequal access to computing resources, and the reproduction of harmful biases in language-model training data.^[22]

These concerns address deployed technologies and their social context. They do not settle whether ASI is feasible or future control failures are likely. They show why assessment cannot be reduced to a future capability threshold: who benefits, who bears costs, and how a system is used matter before that threshold is reached.^[22]

The control problem and safety research

In AI safety, the control problem concerns retaining human influence over highly capable systems. AI alignment addresses intended objectives; AI control can test defenses without assuming alignment. The approaches address related but distinct problems.^[18][19]

Containment, motivation, and monitoring

Stuart Armstrong, Sandberg, and Bostrom's 2012 analysis of an oracle AI, a system intended to answer questions, distinguishes restrictions on capability from methods intended to influence motivation and checks intended to detect failure.^[16]

The paper examines tradeoffs in limiting an oracle's outputs and in asking questions whose answers might themselves be dangerous. It is a theoretical survey of options, not a guarantee that a sufficiently capable system can be contained by any one of them.^[16]

Shutdown incentives and corrigibility

The Off-Switch Game (2017), by Dylan Hadfield-Menell and colleagues, analyzes a simplified interaction between a human and a robot that can disable its shutdown mechanism. A robot committed to a fixed objective can have an incentive to resist interruption because being stopped prevents it from fulfilling that objective.^[17]

In the model, uncertainty about the human's preferences, combined with treating human decisions as information about those preferences, can give the robot an incentive to preserve the switch. The result depends on the model's assumptions, including how the human behaves. It is an argument for investigating designs that remain receptive to correction, not a proof that arbitrary uncertainty makes a real system safe.^[17]

The study also distinguishes instrumental self-preservation from an intrinsic desire to survive. A shutdown incentive can arise from the structure of the assigned task without requiring fear, consciousness, or hostility.^[17]

Scalable oversight and weak-to-strong learning

Human supervision may become less reliable when the work being assessed exceeds the supervisor's expertise. Scalable oversight studies ways of improving evaluation in such settings, including assistance from other systems.^[18][20]

Collin Burns and colleagues' Weak-to-Strong Generalization experiments trained stronger models using labels produced by weaker models. The experiments covered natural-language tasks, chess, and reward modeling. Stronger students sometimes performed better than their weak supervisors, showing that imperfect supervision need not impose a strict ceiling equal to the supervisor's own performance.^[18]

Performance gaps remained, and methods varied in effectiveness across settings. The authors emphasize differences from humans supervising future superhuman systems, including what models had learned during pretraining. The work is a proof of concept, not a solution to superintelligence alignment.^[18]

Empirical control evaluations

Ryan Greenblatt and colleagues' AI Control: Improving Safety Despite Intentional Subversion studies an untrusted code-generating model attempting to introduce exploitable logical errors that pass tests. A weaker trusted model and limited auditing budget are used to test defenses.^[19]

The experiments compare auditing, monitoring, routing suspicious work to trusted systems, and editing. They also investigate coordination between an untrusted monitor and the model it monitors. Tested protocols improved the tradeoff between task performance and avoiding successful attacks relative to simpler baselines.^[19]

These results depend on the programming tasks, model capabilities, red-team strategies, and evaluation budget used. They do not show that the same defenses would control ASI. Their contribution is a way to test control strategies against deliberately adversarial behavior rather than infer safety solely from ordinary helpful behavior.^[19]

Evaluating dangerous capabilities and behavior

Toby Shevlane and colleagues' Model Evaluation for Extreme Risks distinguishes assessments of what a system can do from assessments of whether it tends to behave harmfully. It proposes evaluations connected to decisions about training, deployment, security, and governance, rather than treating evaluation as a one-time benchmark score.^[20]

The paper discusses testing relevant capabilities, including cyber operations, manipulation, and autonomous action, alongside assessments of alignment. It also recognizes limitations: a model's behavior can depend on tools, prompts, access, and the broader system in which it operates. Passing a limited evaluation cannot establish safety in every deployment context.^[20]

For superintelligence research, this distinction prevents two mistaken inferences: a demonstration of a capability does not prove the system will use it maliciously, and a failure to elicit a behavior in one test does not prove that behavior is impossible.^[20]

Debate and governance proposals

Capability measurement and uncertain extrapolation

Francois Chollet's On the Measure of Intelligence (2019) argues that performance at selected tasks is an inadequate measure of general intelligence if it can be obtained through extensive task-specific knowledge or training. He proposes evaluating skill acquisition and generalization relative to prior knowledge and experience, and introduces the ARC benchmark as a way to study some of these questions.^[23]

This position draws attention to the difference between being excellent at a familiar task and learning an unfamiliar one efficiently. It does not prove that artificial superintelligence is impossible, nor does one benchmark supply a complete ASI test. It offers a competing emphasis in measurement: breadth and learning efficiency matter alongside attained performance.^[23]

Risk reduction and restrictions

Eliezer Yudkowsky and Nate Soares' 2025 book If Anyone Builds It, Everyone Dies argues that sufficiently powerful AI would develop conflicting goals, and that a conflict with ASI would be fatal to humanity. Its publisher's description presents this as an urgent argument for changing the course of development. This is the authors' strongly stated prediction, not a demonstrated outcome or a consensus probability estimate.^[32]

The Center for AI Safety's 2023 extinction-risk statement calls for reducing the risk of extinction from AI to be treated as a global priority alongside other major societal risks. Its published signatories include Hinton, Bengio, Demis Hassabis, Sam Altman, and Dario Amodei.^[24]

The statement expresses a policy priority. It does not specify a numerical probability, an expected arrival date, or a claim that an ASI already exists. Agreement with that statement should not be presented as agreement on every proposed route to catastrophe or every proposed restriction.^[24]

The Future of Life Institute's superintelligence initiative advocates prohibiting development until there is broad scientific agreement that it is safe and controllable, and strong public acceptance. Its published materials identify supporters from scientific, political, business, and other communities.^[25]

That position is stronger than calling for risk reduction alone. It is a normative proposal about acceptable development conditions, not an experimental finding about what level of control has been achieved or a survey establishing universal agreement.^[25]

The 2024 Science policy paper by Bengio and coauthors instead sets out a program of technical research and adaptive governance. Proposals include stronger oversight, interpretability, dangerous-capability evaluation, safety cases, and institutions able to respond to changing capabilities. The authors argue that existing harms and prospective extreme risks should both be addressed.^[21]

Timing, benefits, and assumptions

Bostrom's 2026 working paper Optimal Timing for Superintelligence: Mundane Considerations for Existing People analyzes deployment timing under explicit assumptions about benefits, catastrophic risk, and safety progress. It takes a person-affecting perspective and includes potential gains in life expectancy and quality of life as costs of delaying beneficial deployment.^[26]

Its models explore when waiting for additional safety work is desirable and when delay might sacrifice expected benefits. A more detailed version separates reaching a capability level from deploying the resulting system, allowing for a period of testing after a deployable artifact exists.^[26]

The paper is conditional analysis, not a forecast or an estimate of the actual probability of catastrophe. Its conclusions depend on the assumed benefits, risk reductions, and ethical perspective. It illustrates why people can disagree about the preferred pace of development even when they take both potential benefits and serious risks into account.^[26]

Research programs and organizational uses of the term

Organizational announcements identify research objectives; the use of "superintelligence" in a company name or program does not establish a demonstrated capability level.^[27][29][30]

The Future of Humanity Institute at Oxford contributed to earlier work on superintelligence and control, including several papers cited here. It was founded in 2005 and closed on April 16, 2024. It should be described as a historical institution, not a currently operating research center.^[33]

Safe Superintelligence Inc

Safe Superintelligence Inc (SSI) presents safe superintelligence as its single goal and product focus. Its public statement describes safety and capabilities as linked technical problems and says its approach is intended to avoid short-term commercial pressures.^[27]

SSI's own updates report a $1 billion fundraising round in September 2024. In July 2025, Ilya Sutskever announced that he had become CEO, Daniel Levy was president, and Daniel Gross had left. A July 2026 update announced a partnership with NVIDIA intended to increase the company's compute by a factor of ten.^[28]

These statements document the organization's funding, leadership, and plans. They are not independent verification that its safety approach works or that it has produced ASI.^[27][28]

Microsoft AI's humanist superintelligence

On November 6, 2025, Mustafa Suleyman announced Microsoft's MAI Superintelligence Team, led by him. He described humanist superintelligence (HSI) as advanced systems oriented toward specific problems and human benefit, with limits on their autonomy.^[29]

The announcement emphasizes domain-specific applications and contrasts them with an unbounded autonomous entity. It presents medical diagnosis as a directional example and argues for continuing work on containment and alignment. These are Microsoft's intended development principles and claims about its program, not proof that its systems meet the broader ASI definition.^[29]

Microsoft's domain-specific emphasis should not be silently equated with a broad ASI capability claim.^[29]

OpenAI's Superalignment announcement

OpenAI announced Superalignment in July 2023, with Ilya Sutskever and Jan Leike as co-leads. The announcement set a four-year research goal and committed 20% of the compute the company had secured at that point. It proposed building an automated alignment researcher, improving supervision, examining generalization, and stress-testing the alignment process.^[30]

The announcement explicitly distinguished superintelligence from AGI and described success as uncertain. Its concern about supervising more capable systems was a motivation for the research program, not a theorem that every future supervision method must fail. The announced deadline and resource commitment are historical plans; they should not be treated as evidence of present staffing, delivery, or a solved control problem.^[30]

Forecasts and uncertainty

Forecasts need a stated event, population, date, and aggregation method. Predicting human-level task performance, full occupational automation, broad superhuman intelligence, and a societal singularity are different questions.^[31][4]

Katja Grace and colleagues' 2023 survey received responses from 2,778 authors at selected AI venues. Its high-level machine intelligence (HLMI) question concerned machines doing every task better and more cheaply than human workers, excluding activities where being human is intrinsically advantageous. It asked about feasibility, not adoption.^[31]

Assuming scientific activity continued without major negative disruption, the aggregate HLMI distribution, from 1,714 answers, assigned 10% probability by 2027 and 50% by 2047. The occupational-automation question, answered by 774 respondents, produced a much later aggregate 50% date, 2116. The authors discuss framing and interpretation effects.^[31]

These are conditional judgments, not measured probabilities or an ASI schedule. The response rate was approximately 15%, and forecasts varied substantially. The survey does not certify achievement, and its HLMI definition should not silently replace a different ASI definition.^[31]

References

1. ^1 ^2 ^3Bostrom, N. (2003). Ethical Issues in Advanced Artificial Intelligence. In *Cognitive, Emotive and Ethical Aspects of Decision Making in Humans and in Artificial Intelligence*, volume 2, pp. 12-17.
2. ^1 ^2 ^3 ^4Morris, M. R., et al. Position: Levels of AGI for Operationalizing Progress on the Path to AGI. arXiv:2311.02462, version 5, September 24, 2025. Originally presented at ICML 2024.
3. ^1 ^2Good, I. J. (1965). Speculations Concerning the First Ultraintelligent Machine. *Advances in Computers*, 6, pp. 31-88. Virginia Tech's repository provides the introductory extract, including the argument on p. 33.
4. ^1 ^2 ^3 ^4 ^5Vinge, V. (1993). The Coming Technological Singularity: How to Survive in the Post-Human Era. *Vision 21: Interdisciplinary Science and Engineering in the Era of Cyberspace*. NASA Technical Reports Server.
5. ^Bostrom, N. Superintelligence: Paths, Dangers, Strategies. Oxford University Press. First published 2014; linked publisher page describes the paperback edition.
6. ^1 ^2 ^3Bostrom, N., and Roberts, R. (December 1, 2014). Nick Bostrom on Superintelligence. EconTalk. Interview and transcript.
7. ^1 ^2Bostrom, N. (2009). Superintelligence. Contribution to the Edge annual question.
8. ^Bostrom, N. (1998, with later postscripts). How Long Before Superintelligence?. *International Journal of Futures Studies*, 2.
9. ^1 ^2 ^3Sandberg, A., and Bostrom, N. (2008). Whole Brain Emulation: A Roadmap. Future of Humanity Institute Technical Report 2008-3. Full paper in the institute's historical archive.
10. ^1 ^2Shulman, C., and Bostrom, N. (2014). Embryo Selection for Cognitive Enhancement: Curiosity or Game-changer?. *Global Policy*, 5(1), pp. 85-92.
11. ^Davies, G., et al. (2018). Study of 300,486 individuals identifies 148 independent genetic loci influencing general cognitive function. *Nature Communications*, 9, article 2098.
12. ^1 ^2Willett, F. R., et al. (2023). A high-performance speech neuroprosthesis. *Nature*, 620, pp. 1031-1036.
13. ^1 ^2 ^3Bostrom, N. (2012). The Superintelligent Will: Motivation and Instrumental Rationality in Advanced Artificial Agents. *Minds and Machines*, 22(2), pp. 71-85.
14. ^1 ^2Turner, A. M., et al. Optimal Policies Tend to Seek Power. NeurIPS 2021; arXiv version 10 includes subsequent clarifications.
15. ^1 ^2 ^3International AI Safety Report 2026. February 2026. Section 2.2.2, "Loss of control."
16. ^1 ^2 ^3Armstrong, S., Sandberg, A., and Bostrom, N. (2012). Thinking Inside the Box: Controlling and Using an Oracle AI. *Minds and Machines*, 22(4), pp. 299-324.
17. ^1 ^2 ^3Hadfield-Menell, D., Dragan, A., Abbeel, P., and Russell, S. (2017). The Off-Switch Game. IJCAI 2017.
18. ^1 ^2 ^3 ^4Burns, C., et al. (2023). Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervision. arXiv:2312.09390.
19. ^1 ^2 ^3 ^4Greenblatt, R., Shlegeris, B., Sachan, K., and Roger, F. (2024). AI Control: Improving Safety Despite Intentional Subversion. ICML 2024.
20. ^1 ^2 ^3 ^4Shevlane, T., et al. (2023). Model Evaluation for Extreme Risks. arXiv:2305.15324.
21. ^1 ^2 ^3Bengio, Y., et al. (2024). Managing extreme AI risks amid rapid progress. *Science*, 384(6698), pp. 842-845.
22. ^1 ^2University of Washington (March 10, 2021). Large computer language models carry environmental, social risks. Institutional account with statements from the authors of *On the Dangers of Stochastic Parrots*.
23. ^1 ^2Chollet, F. (2019). On the Measure of Intelligence. arXiv:1911.01547.
24. ^1 ^2Center for AI Safety. Statement on AI Extinction Risk. Statement and signatory list, initially released in 2023.
25. ^1 ^2Future of Life Institute. Prominent scientists, faith leaders, policymakers and artists call for a prohibition on superintelligence. Linked press release dated March 27, 2026; campaign statement.
26. ^1 ^2 ^3Bostrom, N. (2026). Optimal Timing for Superintelligence: Mundane Considerations for Existing People. Working paper, version 1.
27. ^1 ^2 ^3Safe Superintelligence Inc. Company statement.
28. ^1 ^2Safe Superintelligence Inc. Updates. Dated updates in September 2024, July 2025, and July 2026.
29. ^1 ^2 ^3 ^4Suleyman, M. (November 6, 2025). Towards Humanist Superintelligence. Microsoft AI.
30. ^1 ^2 ^3Leike, J., and Sutskever, I. (July 5, 2023). Introducing Superalignment. OpenAI.
31. ^1 ^2 ^3 ^4Grace, K., et al. Thousands of AI Authors on the Future of AI. 2023 survey; arXiv:2401.02843, version 3.
32. ^Yudkowsky, E., and Soares, N. (2025). If Anyone Builds It, Everyone Dies: Why Superhuman AI Would Kill Us All. Little, Brown and Company. Publisher's description; publication date September 16, 2025.
33. ^Future of Humanity Institute. Historical archive and institutional history. Documents the institute's operation from 2005 to 2024 and its closure on April 16, 2024.