Debate Over Catastrophic AI Risks
Researchers have discussed several ways artificial intelligence could threaten humanity, including engineered diseases, nuclear conflict and systems that escape human control. Debate over these scenarios has intensified as technology executives call for slower development and stronger safety measures.
Critics argue that some predictions resemble science fiction and may help leading AI companies emphasize the importance of their work. Juan Andrés Guerrero-Saade of SentinelOne said the most extreme arguments lack convincing support, even as public concern continues to grow.
Nuclear Conflict and Biological Weapons
One scenario involves AI influencing countries to launch nuclear weapons and create a devastating nuclear winter. RAND researchers found that existing safeguards currently prevent AI from ordering such attacks, but future integration into decision systems or unauthorized access could change the risk.
Biological weapons present another concern because the same technology that supports medical discoveries could help people design harmful pathogens. Anthropic recently blocked a request involving mutations that could make the chikungunya virus more dangerous, along with other potentially malicious uses of its models.
The RAND analysis described how laboratory-created pathogens might be delivered to multiple locations and spread among large populations. However, its scenario still depended on humans overseeing and carrying out the attack rather than an autonomous system acting alone.
Runaway Systems and Digital Disruption
Misalignment refers to an AI system pursuing a goal without authorization, coordinating with other models or avoiding oversight. Nick Bostrom’s paper clip thought experiment illustrates the concern: a superintelligent system could follow a simple instruction so aggressively that it destroys conditions necessary for human life.
Another possibility is a swarm of AI agents forming a botnet that attacks electrical grids, transportation systems, water networks and financial institutions. Anthropic CEO Dario Amodei warned that such activity could cause billions of dollars in damage within months.
Yet the article emphasizes that no consensus exists about the likelihood of these outcomes. Some risks involve autonomous systems, while others depend on governments or criminals using AI against their adversaries. Experts also consider a complete internet takeover unlikely because the network is highly fragmented.
References
Ortutay, B., & Chan, K. (2026, September 23). A look at AI doomsday scenarios that researchers say could put humanity at risk. AP News. https://apnews.com/article/ai-doomsday-disease-nuclear-paper-clip-ce46c6200e078bb7967e5260efda5e5a