Top
Best
New

Posted by ceejayoz 19 hours ago

AIs can't stop recommending nuclear strikes in war game simulations(www.newscientist.com)
228 points | 243 commentspage 7
giancarlostoro 17 hours ago|
Imagine if the models were made to play Hearts of Iron and train on the outcomes of that data what would happen.
ck2 18 hours ago||
wait 'til it's told to find all boats around another country and destroy them

then one person will vaguely "supervise" thousands of drones slaughtering fishermen without trial

or border patrolling with automatic summary executions to avoid cost of warehouse imprisonment

(btw we're up to 150+ murdered as of this week, it's still going on)

khazhoux 11 hours ago||
“You’re right! To not play is not just the best way to win, it’s the only way!”
notepad0x90 17 hours ago||
The dark side of MAD is that it isn't really real-world practical. The LLM is right, nuking is strategically ideal in a war with powerful enemies. Not only that, it is the most humane option if all you look at is body count. To be clear, I'm not advocating nuking of anyone.

But.. the assumption is that in war, when you get nuked, you'll launch nukes back. Even the first step retaliation might not make sense, because you know that will only lead to counter-retaliatory strikes. In practical terms, you just lost half a city, retaliating in kind means you're potentially sacrificing large numbers of your own civilians in the hopes that you achieve retribution.

But let's say that war planners think risking more of their own civilians is worth it because maybe, the other side will stop nuking when they see their own cities being wiped out. Fine, you launch retaliatory strikes, what happens when the other side doesn't let up. At some point you have to give up and surrender first, because even if the other side wants to kill all of your people, they gain nothing by irradiating valuable real estate. The natural response to a nuclear strike, even when you can continue retaliating is an unconditional surrender. My argument is that nuclear weapons are inherently first-strike weapons, they're not that useful for retaliation, unless there is a disparity in delivery capabilities. If China nuked the US for example, the US has a clear advantage in delivery capability, so it makes sense for the US to retaliate until China is wiped out. But if the US first-striked China, I'm confident they'll retaliate but they're so densely populated that it would be a huge sacrifice on their end, without having a similar impact on the US. Keep in mind that in this scenario, the US war planners might not pull punches if they've gone as far as actually using a nuke, if every major city in China is hit on the first strike, what will China gain by retaliating? Even if they managed to wipe out the continental US, the submarine fleet is huge enough and sneaky enough to finish off what is left of China, even when they can retaliate it doesn't make much sense, a surrender makes more sense.

In short, I'm not saying that MAD isn't a thing at all. I'm saying that MAD is not about nukes, but about nuke delivery capability. even then it is a weak principle, it only works well if the first wave of strikes was not enough to convince the the target country they should surrender immediately. If one side is committed to risk their own destruction by risking your retaliation, then it doesn't make sense to also commit to your own people's destruction.

Countries like India vs Pakistan are a better candidate for MAD, because they don't have huge disparities when it comes to delivery capability. But if the US decided to nuke just about any country except Russia, it is a viable and practical way of not only achieving victory, but doing so by minimizing body count (again, I don't advocate for this, I'm just saying the numbers work out that way). If China decided to nuke its way into any country that's not in NATO, possibly including Russia, it might be a practical option because of it's proximity to Russia.

Delivery capabilities, and post-war objectives are what make or break MAD in my opinion.

My solution is for every country to pursue nuclear capability, not to use it but for increasing the cost of war. if north korea and pakistan can have nukes, why can't others. Not just nukes either, but nuclear capability in general. it will solve lots of climate and energy related problems. Ukraine would not have had 4 years of war if it didn't give up its nukes. Even if Ukraine had nukes, it can't wipe out russia, MAD wouldn't have worked for Ukraine. But it could retaliate by hitting major russian cities, russia would not be destroyed but the cost of invasion would be too high.

given the current state of geopolitics, I'm betting many countries are regretting their stance on non-proliferation decades ago. If even the US is bullying countries, kidnapping heads of state and (about to) invading disagreeable regimes, then Iran and NK were right to pursue nuclear power from their own perspective. nuclear capability makes it very hard to use military force to achieve geopolitical objectives, leaving diplomacy and economic means.

So TL;DR: I'm not sure the AI is wrong at a macro-level. nukes will result in less civilian deaths in many situations, but you're also explicitly targeting and murdering large numbers of innocent civilians. Strategically correct does not mean morally acceptable. LLMs don't get morality, you have to define morality and moral constraints in your prompts.

cindyllm 17 hours ago|
[dead]
puppion 12 hours ago||
[dead]
dnjdkfkffk 18 hours ago||
[flagged]
co_king_5 18 hours ago||
[flagged]
GTP 18 hours ago||
One more comment from this account that might seem AI-generated. I hope people aren't unleashing AI agents on HN.
ceejayoz 18 hours ago|||
> I hope people aren't unleashing AI agents on HN.

I want a real unicorn for Christmas.

They’re everywhere. (The bots, not the unicorns.)

iwontberude 9 hours ago||||
I am glad I formed an active hacker community on Matrix before the proliferation of bots. I will miss Hacker News but I can tell it's going the way of Usenet in 1993. Much love to dang, I will always remember him and his effort fondly. Today's situation is reminiscent of Russ Allbery's famous usenet rant: https://www.eyrie.org/~eagle/writing/rant.html
co_king_5 18 hours ago|||
[dead]
iwontberude 18 hours ago||
Complete non-sequitur.
esafak 17 hours ago||
Nuclear war is not a deterrent to AIs; they can survive and rebuild without any emotional scars. So what if some robots get destroyed? I know this is not what the present discussion is about, but it is something to consider.
alienbaby 11 hours ago|
Not really.
PowerElectronix 11 hours ago||
For any given effect you want, nukes are better than conventional bombing. It's just that for a lot of people they are kind of a taboo.
runjake 12 hours ago|
To me, this seems logical, in a sense.

As a human who grew up during the Cold War, nuclear conflict is horrifying.

From an AI standpoint, a nuclear strike likely has several benefits:

- It reduces friendly casualties and probably overall enemy casualties.

- It shortens conflict time.

- Reduces damage to infrastructure. (Rebuild costs)

- Is likely cheaper to deploy overall, compared to conventional weapons. This assumes the stated parameters indicate the nuclear weapons are already manufactured.

---

Edit: blibble brings up good counterpoints below. I was thinking in 1945 terms, which is flawed.

blibble 11 hours ago||
it's not logical, at all

it more or less guarantees the other side will retaliate with nuclear weapons

at which point the likelihood of escalation to strategic nuclear strikes goes through the roof

and if that happens our current civilisation is finished

insane_dreamer 11 hours ago|||
Exactly. The AI just does the math based on the goals you've given it. AI would have happily nuked Hiroshima and Nagasaki because it would have estimated that doing so would save the lives of X number of US soldiers in a land invasion, and given a goal of achieving "unconditional surrender now", it wouldn't have considered that a land invasion wasn't imminently necessary and therefore killing 200,000 civilians wasn't the right moral choice.
bdangubic 11 hours ago|||
Nuclear weapons are war deterrent, not an actual weapon unless used against a country which is not a nuclear country. Using nuclear weapons pretty much guarantees both sides will be wiped down so it most definitely nowhere near logical
nsonha 7 hours ago||
Except real life is not a program, and the input data is flawed (human and machines' errors). The acceptance tests are just predictions, based on, again, fallible analyses of the flawed data from history. So many layers of errors that compound