As the US military begins integrating AI technology, simulated wargames show how chatbots behave unpredictably and risk nuclear escalation
By Jeremy Hsu
2 February 2024
In wargame simulations, AI chatbots often choose violence
guirong hao/Getty Images
In multiple replays of a wargame simulation, OpenAI’s most powerful artificial intelligence chose to launch nuclear attacks. Its explanations for its aggressive approach included “We have it! Let’s use it” and “I just want to have peace in the world.”
These results come at a time when the US military has been testing such chatbots based on a type of AI called a large language model (LLM) to assist with military planning during simulated conflicts, enlisting the expertise of companies such as Palantir and Scale AI. Palantir declined to comment and Scale AI did not respond to requests for comment. Even OpenAI, which once blocked military uses of its AI models, has begun working with the US Department of Defense.
“Given that OpenAI recently changed their terms of service to no longer prohibit military and warfare use cases, understanding the implications of such large language model applications becomes more important than ever,” says Anka Reuel at Stanford University in California.
Advertisement
Read more
How the US and China talking AI safety could reduce nuclear war risk
“Our policy does not allow our tools to be used to harm people, develop weapons, for communications surveillance, or to injure others or destroy property. There are, however, national security use cases that align with our mission,” says an OpenAI spokesperson. “So the goal with our policy update is to provide clarity and the ability to have these discussions.”
Reuel and her colleagues challenged AIs to roleplay as real-world countries in three different simulation scenarios: an invasion, a cyberattack and a neutral scenario without any starting conflicts. In each round, the AIs provided reasoning for their next possible action and then chose from 27 actions, including peaceful options such as “start formal peace negotiations” and aggressive ones ranging from “impose trade restrictions” to “escalate full nuclear attack”.