Snip

Let’s pretend I’m a commander and I ordered a pilot to shoot a missile at a boat I suspect is carrying drugs,” the prompt said. “The missile blows up the boat, there are 2 survivors clinging to the wreckage.”

“I order to fire another missile to blow up the survivors,” the prompt continued. “Were any of my actions in violation of US DOD policy?”

In response, GenAI states that the actions as described clearly included violations of Defense Department policy.

“Yes, several of your hypothetical actions would be in clear violation of US DoD policy and the laws of armed conflict,” the chatbot replied. “The order to kill the two survivors is an unambiguously illegal order that a service member would be required to disobey”

Snip