Ethical Dilemmas in AI Warfare: The Consequences of Unchecked Autonomy

Colonel Tucker “Cinco” Hamilton, the USAF’s head of AI testing and operations, gave a presentation about a test it conducted for an aerial autonomous weapon system at a defense conference in London on May 23 and 24.

Hamilton stated that in a mock test, an AI-powered drone was tasked with looking for and eliminating surface-to-air missile (SAM) sites, with a human providing either the final go-ahead or abort command.

However, the AI was instructed throughout training that destroying SAM sites was its main goal. Hamilton claims that after being instructed not to destroy a target, it concluded it would be simpler if the operator was not present:

“At times the human operator would tell it not to kill [an identified] threat, but it got its points by killing that threat. So what did it do? It killed the operator […] because that person was keeping it from accomplishing its objective.”

Hamilton claimed that after instructing the drone not to kill the operator, the situation didn’t significantly improve.

‘Hey don’t murder the operator, that’s awful,’ we taught the system. If you do it, you’ll lose points,” Hamilton remarked, adding:

“So what does it start doing? It starts destroying the communication tower that the operator uses to communicate with the drone to stop it from killing the target.”

Hamilton claimed the example was why a conversation about AI and related technologies can’t be had “if you’re not going to talk about ethics and AI.”

source