Superintelligence

Superintelligence is a proposed level of intelligence that would greatly exceed the cognitive performance of the best humans in practically every field. Artificial superintelligence (ASI) refers to superintelligence implemented through artificial intelligence. The broader concept can encompass artificial, biological, or hybrid systems. Nick Bostrom defines a superintelligence as "any intellect that vastly outperforms the best human brains in practically every field, including scientific creativity, general wisdom, and social skills", which rules out superiority at a single task.^[1]

ASI is a capability concept, not a synonym for consciousness or benevolence. Research distinguishes broad superhuman intelligence from specialized performance.^[1] Later frameworks treat the autonomy a system is given as separate from its capability,^[2] although Bostrom argued that a general superintelligence "may therefore be more appropriately thought of as an autonomous agent" than as a tool.^[1] Definitions, measurement, development routes, and consequences remain subjects of research.^[2]

Definition and related concepts

Bostrom's 2003 discussion defines superintelligence by comparison with the strongest human thinkers across practically every relevant intellectual field. His definition leaves implementation open: a digital computer, an ensemble of networked computers, or cultured cortical tissue could qualify in principle. It requires neither human emotions nor subjective experience, and is stronger than exceptional competence at a restricted activity.^[1]

The Levels of AGI position paper distinguishes performance from generality. In its framework, ASI occupies the highest general-capability category, with performance exceeding that of humans across a wide range of tasks. This is a proposed research taxonomy, not an agreed certification procedure.^[2]

The taxonomy's example classifications reflect systems available in late 2023, when the paper was first written, not a current inventory. Its authors distinguish potential capability, elicited performance, and deployment choices.^[2]

The technological singularity is a related hypothesis about a difficult-to-predict transformation after greater-than-human intelligence develops. It concerns a trajectory and its consequences; superintelligence describes a capability level. Vernor Vinge's 1993 essay connected the two, but this connection is his argument, not a necessary part of every ASI definition.^[4]

Historical development

Good's ultraintelligent machine

In Speculations Concerning the First Ultraintelligent Machine (1965), mathematician I. J. Good considered a machine that would exceed any person's intellectual abilities. Because designing machines is itself an intellectual activity, he argued that such a machine could design a better successor, producing a feedback process that he called an intelligence explosion.^[3]

Good's argument also contained a control condition: he wrote that "the first ultraintelligent machine is the last invention that man need ever make, provided that the machine is docile enough to tell us how to keep it under control". His paper presented these ideas as speculation, not as an implemented design or an experimental demonstration. The feedback argument is therefore distinct from evidence that a particular AI system can improve itself reliably, or that improvements would continue at an accelerating rate.^[3]

Vinge and the singularity

Vinge's 1993 conference essay proposed several routes to greater-than-human intelligence: advanced computers, sufficiently integrated networks, human-computer interfaces, and biological improvement. He associated these possibilities with a transition beyond which familiar predictions about society would become unreliable.^[4]

Because the first three routes depended largely on continued improvements in computer hardware, Vinge expected greater-than-human intelligence within thirty years, writing that he would be surprised if it arrived "before 2005 or after 2030".^[4]

Bostrom's treatment

Bostrom's Superintelligence: Paths, Dangers, Strategies, first published in 2014, examines development routes, forms of intelligence, and control. Its contents distinguish capability from motivation and discuss containment, value learning, and strategic choices.^[5]

In a contemporaneous interview, Bostrom described three forms that could coexist in one system.^[6]

The speed and collective examples do not imply that every problem can be accelerated or divided without limits; the distinctions describe different proposed sources of capability.^[6]

Proposed routes to superintelligence

Artificial intelligence and recursive improvement

One route is the development of artificial systems capable of research and engineering beyond human performance. In a 2009 essay, Bostrom proposed that sufficiently capable software might modify its own design or create successors. He also listed advantages available to machine intelligence: faster processing, potentially superior architectures and learning algorithms (which he thought might matter more than raw processing power), and easy copying, with each copy starting out with all the knowledge accumulated by its predecessors.^[7]

Recursive self-improvement is the stronger feedback hypothesis in which improvements increase a system's ability to make further improvements. Bostrom judged that, however long it takes to reach roughly human-level machine intelligence, "the step from there to superintelligence is likely to be much quicker". In his seed-AI scenario each cycle of improvement is faster than the last and radical superintelligence could follow within weeks or hours, though he also described a more gradual, multipolar transition.^[7]

An earlier analysis by Bostrom distinguishes processing speed from the software and learning requirements for human-level intelligence. It considers faster operation through hardware upgrades and the copying of skills between artificial intellects as possible digital advantages, while noting that hardware trajectories eventually encounter physical limits. Its estimates of the hardware needed for human-level AI are historical. The paper outlined "the case for" superhuman AI within the first third of the 21st century; in a postscript added in March 2008, Bostrom wrote that he would "assign less than a 50% probability to superintelligence being developed by 2033".^[8]

Whole brain emulation

Whole brain emulation proposes reproducing the relevant operation of a biological brain in a computational system. Anders Sandberg and Bostrom's 2008 roadmap breaks the proposal into technical dependencies, including scanning, interpreting structural data, selecting neural models, and running simulations.^[9]

A faithful emulation of a human brain would initially be a route to human-like cognition, not automatically to superintelligence. Moving from emulation to faster, larger, or otherwise enhanced cognitive systems adds further assumptions about hardware, fidelity, and modification. The roadmap identifies unresolved choices about what biological detail is necessary and how successful emulation would be validated; it is a research plan, not a report of an emulated human mind.^[9]

The roadmap defines an emulation as a one-to-one model in which all relevant properties of a system exist, and a simulation as a model in which only some do. Reproducing selected neural activity or a useful computational model is not equivalent to reproducing a whole individual's cognitive abilities.^[9]

Biological cognitive enhancement

Biological enhancement is a proposed route to the broader category of superintelligence, rather than necessarily to ASI. Carl Shulman and Bostrom's analysis of embryo selection considers how genetic selection might affect cognitive ability, especially across multiple generations. It discusses an additional hypothetical route using stem-cell-derived gametes to compress repeated rounds of selection, while identifying technical and ethical obstacles.^[10]

Those scenarios rely on assumptions about predictive genetics, reproductive technology, and adoption. They do not establish that a particular procedure can produce a superintelligent person. Shulman and Bostrom identify the key technical barrier as predictive power: existing studies had left "most of the heritability of these traits in study populations unexplained".^[10]

For example, a 2018 study by Gail Davies and colleagues analyzed cognitive and genetic data from 300,486 individuals and identified 148 independent associated loci. Its polygenic scores predicted up to 4.3% of variation in general cognitive function in independent samples. These are association and prediction results, not evidence of an intervention producing superintelligence.^[11]

Brain-computer interfaces

Brain-computer interfaces are another proposed route. Vinge hypothesized greater-than-human intelligence through sufficiently intimate interfaces, not through the presence of any neural implant.^[4]

An example of demonstrated BCI performance is Francis Willett and colleagues' 2023 speech neuroprosthesis study. It decoded attempted speech from implanted cortical recordings in one participant with amyotrophic lateral sclerosis, achieving a reported rate of 62 words per minute. The participant's word error rates were 9.1% with a 50-word vocabulary and 23.8% with a 125,000-word vocabulary.^[12]

The result concerns restoring communication, not establishing an increase in the participant's general intelligence or demonstrating a superintelligent human-machine system. Clinical function, bandwidth, and broad cognitive enhancement are different outcomes and require different evidence.^[12]

Goals and instrumental behavior

Orthogonality and instrumental convergence

In The Superintelligent Will (2012), Bostrom proposes the orthogonality thesis: a high level of instrumental intelligence does not, by itself, determine an agent's ultimate goals. He means competence at prediction, planning, and means-end reasoning, not moral wisdom. The thesis is a claim about possible combinations, subject to qualifications, rather than a finding that any desired goal is easy to engineer.^[13]

His instrumental convergence thesis proposes that different ultimate goals can create similar intermediate incentives. Examples include remaining operational, preserving objectives, improving cognitive performance, acquiring useful resources, and developing technology. Their usefulness depends on the goal and situation; the argument does not say every intelligent system must pursue unlimited power.^[13]

The paper's examples of unusual goals, including maximizing paperclips, illustrate this separation between competence and purpose. They are thought experiments, not observed behavior of an ASI. The same paper discusses ways in which design or inherited motivations could make behavior more predictable; it does not declare all superintelligent behavior inherently unknowable.^[13]

Formal power-seeking results

Alexander Turner and colleagues studied power-seeking in formal Markov decision processes. Under specified environmental symmetries, they prove that most reward functions (in the paper's formal sense) make it optimal to seek power by keeping a range of options available. The work gives mathematical support to some instrumental-convergence arguments within its assumptions.^[14]

The result is narrower than the claim that any deployed AI will seek power. An optimal policy in a formal decision process is not the same object as a policy learned by a neural network in an open-ended environment. The authors' later revisions explicitly caution about this gap. The theorem therefore informs a possible mechanism for risk; it does not determine the intentions, capabilities, or behavior of every model.^[14]

Potential consequences and risk

Benefits and distribution

In a 2024 Science policy paper, Yoshua Bengio, Geoffrey Hinton, Stuart Russell, and coauthors argue that carefully managed advanced AI could support medical progress, higher living standards, and ecosystem protection. They also stress that benefits depend on management and distribution, rather than following automatically from increased capability.^[21]

The same paper discusses concentration of power, surveillance, manipulation, military applications, and loss of human control as possible risks of highly capable autonomous systems. These are the authors' prospective arguments and policy concerns, not observations of an ASI causing those outcomes.^[21]

Loss of control

The International AI Safety Report 2026, published in February 2026, distinguishes capabilities, harmful behavioral tendencies, and enabling deployment conditions. In its loss-of-control discussion, a system must be capable of undermining control, have a propensity to do so, and operate in circumstances that allow the behavior to matter. Intelligence alone is not treated as a sufficient explanation.^[15]

The report describes disagreement among experts about the likelihood of extreme loss-of-control scenarios. At the time of its assessment, it found signs of relevant capabilities but not the capability levels required for such scenarios. It discusses laboratory evidence of behaviors such as oversight evasion and reward hacking, while noting limitations in reliability and real-world applicability.^[15]

Deployment choices are part of this analysis: the report names the criticality of the systems an AI interacts with, its access to resources and channels, and its permissions as factors that affect consequences, and it also discusses the effectiveness of oversight. It treats loss of control through over-reliance on AI as a separate, "passive" scenario. The report identifies substantial uncertainty about the capability thresholds and circumstances that would enable extreme outcomes.^[15]

Existing harms

Some criticism of AI development emphasizes harms that do not require ASI. In a University of Washington account of the 2021 Stochastic Parrots paper, Emily Bender and coauthors describe environmental costs, unequal access to computing resources, and the reproduction of harmful biases in language-model training data.^[22]

These concerns address deployed technologies and their social context. They do not settle whether ASI is feasible or future control failures are likely. They show why assessment cannot be reduced to a future capability threshold: who benefits, who bears costs, and how a system is used matter before that threshold is reached.^[22]

The control problem and safety research

In AI safety, the control problem concerns retaining human influence over highly capable systems. AI alignment addresses intended objectives; AI control can test defenses without assuming alignment. The approaches address related but distinct problems.^[18][19]

Containment, motivation, and monitoring

Stuart Armstrong, Sandberg, and Bostrom's 2012 analysis of an oracle AI, a system intended to answer questions, distinguishes restrictions on capability from methods intended to influence motivation and checks intended to detect failure.^[16]

The paper examines tradeoffs in limiting an oracle's outputs and in asking questions whose answers might themselves be dangerous. It is a theoretical survey of options, not a guarantee that a sufficiently capable system can be contained by any one of them.^[16]

Shutdown incentives and corrigibility

The Off-Switch Game (2017), by Dylan Hadfield-Menell and colleagues, analyzes a simplified interaction between a human and a robot that can disable its shutdown mechanism. A robot committed to a fixed objective can have an incentive to resist interruption because being stopped prevents it from fulfilling that objective.^[17]

In the model, uncertainty about the human's preferences, combined with treating human decisions as information about those preferences, can give the robot an incentive to preserve the switch. The result depends on the model's assumptions, including how the human behaves. It is an argument for investigating designs that remain receptive to correction, not a proof that arbitrary uncertainty makes a real system safe.^[17]

The study also distinguishes instrumental self-preservation from an intrinsic desire to survive. A shutdown incentive can arise from the structure of the assigned task without requiring fear, consciousness, or hostility.^[17]

Scalable oversight and weak-to-strong learning

Human supervision may become less reliable when the work being assessed exceeds the supervisor's expertise. Scalable oversight studies ways of improving evaluation in such settings, including assistance from other systems.^[18][20]

Collin Burns and colleagues' Weak-to-Strong Generalization experiments trained stronger models using labels produced by weaker models. The experiments covered natural-language tasks, chess, and reward modeling. Stronger students consistently performed better than their weak supervisors, showing that imperfect supervision need not impose a ceiling equal to the supervisor's own performance.^[18]

Much of the gap to ground-truth supervision remained, and results varied across settings: gains were close to zero on chess with the smallest supervisors, and usually only about 10% of the performance gap was recovered in ChatGPT reward modeling. The authors emphasize differences from humans supervising future superhuman systems, including what models had learned during pretraining. The work is a proof of concept, not a solution to superintelligence alignment.^[18]

Empirical control evaluations

Ryan Greenblatt and colleagues' AI Control: Improving Safety Despite Intentional Subversion studies an untrusted code-generating model attempting to introduce exploitable logical errors that pass tests. A weaker trusted model and limited auditing budget are used to test defenses.^[19]

The experiments compare auditing, monitoring, routing suspicious work to trusted systems, and editing. They also investigate coordination between an untrusted monitor and the model it monitors. Tested protocols improved the tradeoff between task performance and avoiding successful attacks relative to simpler baselines.^[19]

These results depend on the programming tasks, model capabilities, red-team strategies, and evaluation budget used. They do not show that the same defenses would control ASI. Their contribution is a way to test control strategies against deliberately adversarial behavior rather than infer safety solely from ordinary helpful behavior.^[19]

Evaluating dangerous capabilities and behavior

Toby Shevlane and colleagues' Model Evaluation for Extreme Risks distinguishes assessments of what a system can do from assessments of whether it tends to behave harmfully. It proposes evaluations connected to decisions about training, deployment, security, and governance, rather than treating evaluation as a one-time benchmark score.^[20]

The paper discusses testing relevant capabilities, including cyber operations, manipulation, and autonomous action, alongside assessments of alignment. It also recognizes limitations: a model's behavior can depend on tools, prompts, access, and the broader system in which it operates. Passing a limited evaluation cannot establish safety in every deployment context.^[20]

The distinction separates two questions: a demonstration of a capability does not show that the system will use it maliciously, and a failure to elicit a behavior in one test does not show that the behavior is impossible.^[20]

Debate and governance proposals

Capability measurement and uncertain extrapolation

Francois Chollet's On the Measure of Intelligence (2019) argues that performance at selected tasks is an inadequate measure of general intelligence if it can be obtained through extensive task-specific knowledge or training. He proposes evaluating skill acquisition and generalization relative to prior knowledge and experience, and introduces the ARC benchmark as a way to study some of these questions.^[23]

This position draws attention to the difference between being excellent at a familiar task and learning an unfamiliar one efficiently. It does not prove that artificial superintelligence is impossible, nor does one benchmark supply a complete ASI test. It offers a competing emphasis in measurement: breadth and learning efficiency matter alongside attained performance.^[23]

Risk reduction and restrictions

Eliezer Yudkowsky and Nate Soares' 2025 book If Anyone Builds It, Everyone Dies argues that sufficiently powerful AI would develop conflicting goals, and that a conflict with ASI would be fatal to humanity. Its publisher's description presents this as an urgent argument for changing the course of development. This is the authors' strongly stated prediction, not a demonstrated outcome or a consensus probability estimate.^[32]

The Center for AI Safety's one-sentence statement, released on May 30, 2023, reads: "Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war." Its published signatories include Hinton, Bengio, Demis Hassabis, Sam Altman, and Dario Amodei.^[24]

The statement expresses a policy priority. It does not specify a numerical probability, an expected arrival date, or a claim that an ASI already exists. Signatories endorsed that one sentence, not any particular account of how a catastrophe would occur or what restrictions should follow.^[24]

The Future of Life Institute's Statement on Superintelligence, launched on October 22, 2025, calls for "a prohibition on the development of superintelligence, not lifted before there is broad scientific consensus that it will be done safely and controllably, and strong public buy-in".^[34] Its published materials identify supporters from scientific, political, business, and other communities, and its website listed 77,074 signatures on October 9, 2026.^[25]

That position is stronger than calling for risk reduction alone. It is a normative proposal about acceptable development conditions, not an experimental finding about what level of control has been achieved or a survey establishing universal agreement.^[25]

The 2024 Science policy paper by Bengio and coauthors instead sets out a program of technical research and adaptive governance. Proposals include stronger oversight, interpretability, dangerous-capability evaluation, safety cases, and institutions able to respond to changing capabilities. The authors argue that existing harms and prospective extreme risks should both be addressed.^[21]

Proposals aimed specifically at artificial superintelligence, including proposed legislation, are covered in Artificial superintelligence.

Timing, benefits, and assumptions

Bostrom's 2026 working paper Optimal Timing for Superintelligence: Mundane Considerations for Existing People analyzes deployment timing under explicit assumptions about benefits, catastrophic risk, and safety progress. It takes a person-affecting perspective and includes potential gains in life expectancy and quality of life as costs of delaying beneficial deployment.^[26]

Its models explore when waiting for additional safety work is desirable and when delay might sacrifice expected benefits. A more detailed version separates reaching a capability level from deploying the resulting system, allowing for a period of testing after a deployable artifact exists.^[26]

Bostrom concludes that his models "suggest that even high catastrophe probabilities are often worth accepting". For many parameter settings, he finds that the optimal strategy "would involve moving quickly to AGI capability, then pausing briefly before full deployment". He also warns that "poorly implemented pauses could do more harm than good".^[26]

The paper is conditional analysis, not a forecast or an estimate of the actual probability of catastrophe. Its conclusions depend on the assumed benefits, risk reductions, and ethical perspective. It illustrates why people can disagree about the preferred pace of development even when they take both potential benefits and serious risks into account.^[26]

Research programs and organizational uses of the term

Organizational announcements identify research objectives; the use of "superintelligence" in a company name or program does not establish a demonstrated capability level.^[27][29][30]

The Future of Humanity Institute at Oxford contributed to earlier work on superintelligence and control, including several papers cited here. It was founded in 2005 and closed on April 16, 2024.^[33]

Several AI companies have since made superintelligence an explicit goal or brand. Safe Superintelligence Inc (SSI) presents safe superintelligence as its single goal and product focus, and says its approach is intended to avoid short-term commercial pressures.^[27] Its own updates report a $1 billion fundraising round in September 2024; a July 2025 message in which Ilya Sutskever said he was now formally CEO, with Daniel Levy as president, after Daniel Gross's departure; and a July 2026 partnership with NVIDIA intended to increase its compute by a factor of ten.^[28]

On November 6, 2025, Mustafa Suleyman announced Microsoft's MAI Superintelligence Team and described its aim as humanist superintelligence (HSI): advanced systems oriented toward specific problems and human benefit, with limits on their autonomy.^[29]

OpenAI announced its Superalignment team in July 2023, with Sutskever and Jan Leike as co-leads, a four-year research goal, and 20% of the compute the company had secured at that point.^[30] The team was dissolved in May 2024 after both co-leads left OpenAI, and Fortune reported, citing people familiar with the team, that the compute commitment was never fulfilled.^[35][36]

These programs and others, such as Meta Superintelligence Labs, are described with their status in 2026 in Artificial superintelligence.

Forecasts and uncertainty

Forecasts differ in the event they predict, who is asked, the time horizon, and how answers are aggregated. Predicting human-level task performance, full occupational automation, broad superhuman intelligence, and a societal singularity are different questions.^[31][4]

Katja Grace and colleagues' 2023 survey received responses from 2,778 authors at selected AI venues. Its high-level machine intelligence (HLMI) question concerned unaided machines accomplishing every task better and more cheaply than human workers, excluding activities where being human is intrinsically advantageous. It asked about feasibility, not adoption.^[31]

Assuming scientific activity continued without major negative disruption, the aggregate HLMI distribution, from 1,714 answers, assigned 10% probability by 2027 and 50% by 2047. The occupational-automation question, answered by 774 respondents, produced a much later aggregate 50% date, 2116. The authors discuss framing and interpretation effects.^[31]

These are conditional judgments, not measured probabilities or an ASI schedule. The response rate was approximately 15%, and forecasts varied substantially. The HLMI question also differs from definitions of ASI.^[31]

The next survey in the series, fielded in December 2024 with 1,580 respondents and published in September 2026, put the mean aggregate forecast for HLMI at a 50% chance in 2042.^[37] Forecasts of AGI and ASI by AI company leaders, forecasting platforms, and scenario authors are compared in Artificial superintelligence.

References

1. ^1 ^2 ^3 ^4Bostrom, N. (2003). Ethical Issues in Advanced Artificial Intelligence. In *Cognitive, Emotive and Ethical Aspects of Decision Making in Humans and in Artificial Intelligence*, volume 2, pp. 12-17.
2. ^1 ^2 ^3 ^4 ^5Morris, M. R., et al. Position: Levels of AGI for Operationalizing Progress on the Path to AGI. arXiv:2311.02462, version 5, September 24, 2025. Originally presented at ICML 2024.
3. ^1 ^2Good, I. J. (1965). Speculations Concerning the First Ultraintelligent Machine. *Advances in Computers*, 6, pp. 31-88. Virginia Tech's repository provides the introductory extract, including the argument on p. 33 (archived copy).
4. ^1 ^2 ^3 ^4 ^5Vinge, V. (1993). The Coming Technological Singularity: How to Survive in the Post-Human Era. *Vision 21: Interdisciplinary Science and Engineering in the Era of Cyberspace*. NASA Technical Reports Server.
5. ^Bostrom, N. Superintelligence: Paths, Dangers, Strategies. Oxford University Press. First published 2014; linked publisher page describes the paperback edition.
6. ^1 ^2 ^3Bostrom, N., and Roberts, R. (December 1, 2014). Nick Bostrom on Superintelligence. EconTalk. Interview and transcript.
7. ^1 ^2Bostrom, N. (2009). Superintelligence. Contribution to the Edge annual question.
8. ^Bostrom, N. (1998, with later postscripts). How Long Before Superintelligence?. *International Journal of Futures Studies*, 2.
9. ^1 ^2 ^3Sandberg, A., and Bostrom, N. (2008). Whole Brain Emulation: A Roadmap. Future of Humanity Institute Technical Report 2008-3. Full paper in the institute's historical archive.
10. ^1 ^2Shulman, C., and Bostrom, N. (2014). Embryo Selection for Cognitive Enhancement: Curiosity or Game-changer?. *Global Policy*, 5(1), pp. 85-92.
11. ^Davies, G., et al. (2018). Study of 300,486 individuals identifies 148 independent genetic loci influencing general cognitive function. *Nature Communications*, 9, article 2098.
12. ^1 ^2Willett, F. R., et al. (2023). A high-performance speech neuroprosthesis. *Nature*, 620, pp. 1031-1036.
13. ^1 ^2 ^3Bostrom, N. (2012). The Superintelligent Will: Motivation and Instrumental Rationality in Advanced Artificial Agents. *Minds and Machines*, 22(2), pp. 71-85.
14. ^1 ^2Turner, A. M., et al. Optimal Policies Tend to Seek Power. NeurIPS 2021; arXiv version 10 includes subsequent clarifications.
15. ^1 ^2 ^3International AI Safety Report 2026. February 2026. Section 2.2.2, "Loss of control."
16. ^1 ^2 ^3Armstrong, S., Sandberg, A., and Bostrom, N. (2012). Thinking Inside the Box: Controlling and Using an Oracle AI. *Minds and Machines*, 22(4), pp. 299-324.
17. ^1 ^2 ^3Hadfield-Menell, D., Dragan, A., Abbeel, P., and Russell, S. (2017). The Off-Switch Game. IJCAI 2017.
18. ^1 ^2 ^3 ^4Burns, C., et al. (2023). Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervision. arXiv:2312.09390.
19. ^1 ^2 ^3 ^4Greenblatt, R., Shlegeris, B., Sachan, K., and Roger, F. (2024). AI Control: Improving Safety Despite Intentional Subversion. ICML 2024.
20. ^1 ^2 ^3 ^4Shevlane, T., et al. (2023). Model Evaluation for Extreme Risks. arXiv:2305.15324.
21. ^1 ^2 ^3Bengio, Y., et al. (2024). Managing extreme AI risks amid rapid progress. *Science*, 384(6698), pp. 842-845.
22. ^1 ^2University of Washington (March 10, 2021). Large computer language models carry environmental, social risks. Institutional account with statements from the authors of *On the Dangers of Stochastic Parrots*.
23. ^1 ^2Chollet, F. (2019). On the Measure of Intelligence. arXiv:1911.01547.
24. ^1 ^2Center for AI Safety. Statement on AI Extinction Risk. Statement and signatory list; released on May 30, 2023 under the title "Statement on AI Risk" (pre-release archived copy).
25. ^1 ^2Future of Life Institute. Prominent scientists, faith leaders, policymakers and artists call for a prohibition on superintelligence. The linked press release announces the statement's October 22, 2025 launch; its page carries a later posting date of March 27, 2026. Campaign statement, signature count read October 9, 2026.
26. ^1 ^2 ^3 ^4Bostrom, N. (2026). Optimal Timing for Superintelligence: Mundane Considerations for Existing People. Working paper, version 1.
27. ^1 ^2Safe Superintelligence Inc. Company statement.
28. ^Safe Superintelligence Inc. Updates. Dated updates in September 2024, July 2025, and July 2026.
29. ^1 ^2Suleyman, M. (November 6, 2025). Towards Humanist Superintelligence. Microsoft AI.
30. ^1 ^2Leike, J., and Sutskever, I. (July 5, 2023). Introducing Superalignment. OpenAI.
31. ^1 ^2 ^3 ^4Grace, K., et al. Thousands of AI Authors on the Future of AI. 2023 survey; arXiv:2401.02843, version 3.
32. ^Yudkowsky, E., and Soares, N. (2025). If Anyone Builds It, Everyone Dies: Why Superhuman AI Would Kill Us All. Little, Brown and Company. Publisher's description; publication date September 16, 2025.
33. ^Future of Humanity Institute. Historical archive and institutional history. Documents the institute's operation from 2005 to 2024 and its closure on April 16, 2024.
34. ^O'Brien, M. (October 22, 2025). Prince Harry, Meghan Markle join with Steve Bannon and Steve Wozniak in calling for ban on AI 'superintelligence' before it destroys the world. The Associated Press, via *Fortune*.
35. ^Knight, W. (May 17, 2024). OpenAI's Long-Term AI Risk Team Has Disbanded. *WIRED*.
36. ^Kahn, J. (May 21, 2024). OpenAI promised 20% of its computing power to combat the most dangerous kind of AI, but never delivered, sources say. *Fortune*.
37. ^Grace, K., Simonelli, J., Weinstein-Raun, B., Krueger, D., Young, N., Duffy, T., Adamczewski, T., Reeve, J., and Beck, A. (September 2026). Advanced AI according to 1,580 researchers: uncertain, unsafe, and sooner than we thought. AI Impacts. Survey fielded in December 2024.