Top
Best
New

Posted by realsarm 4 days ago

US Military had close call after using AI for hallucinated intelligence report(www.cnn.com)
518 points | 394 commentspage 2
cloudie78 4 days ago|
At least it was only a close call, unlike last time: https://en.wikipedia.org/wiki/2026_Minab_school_attack
ck2 4 days ago||
forget "close calls" there's been actual murder

just a reminder "AI" selected the elementary school for bombing that murdered over 250 kids

the intel was outdated but that's no excuse because they ended the division that reviewed targets by hand otherwise

if Iran murdered 250 US school kids he'd turn the entire country to sand

vintermann 4 days ago|
> just a reminder "AI" selected the elementary school for bombing that murdered over 250 kids

I think that's almost certainly just AI washing. They didn't do it because the AI told them to do it, they did it because they wanted to do it, and the AI is an excuse.

It could be that they had an AI and told it to come to the conclusion they wanted. But more likely there was no AI at all.

ck2 4 days ago||
I'm clearly no fan of this administration or US Military but I don't believe for a minute they purposely wanted to murder 250 school kids

if I remember the reporting correctly there were like 1000+ targets picked for simultaneous bombing by Tomahawks (at $4 Million a pop) and the division that reviews targets had been dismantled by this administration so the "AI" list was never double-checked

apathy vs malice

though the final reason doesn't matter to those kids or their families

Iran is going to end like Iraq and Afghanistan and Vietnam and Korea way before that, we just finally leave because we can't "win" and leave it worse than the horror it was in the first place

I don't even think the Dems can change it if they somehow win the Senate, it's going to be this nightmare through 2029

vintermann 4 days ago|||
It's always like that - no matter what they do, you refuse to believe that "the boys in uniform" aren't basically still good and decent people.

How do you explain the double tap?

puelocesar 4 days ago||
I always said that AI could cause human extinction not by being too smart, but by doing dumb things because some stupid human overestimated its capabilities and used it where it should not.
muchdoubt 3 days ago||
It’s possible that they’re hiding that the friendly fire incident in Kuwait in March 2026 (3 jets destroyed, $250mm+ loss) was a real incident caused by AI. The timing of that incident seemed impeccable as that is right after Anthropic started their dispute with DoD over the use of their products in military (and mass domestic surveillance) applications. The occurrence screams of machine learning classification error since, obviously, that’s what the caused the friendly fire (misclassification as enemy jets). Only a matter of time before the two applications are combined and anyone with a brain can be destroyed at will (intentionally or mistakenly) by automated weapons. This is what citizens of this world should really be fearing. (This is not my original opinion, but one that is already out there, so please don’t target me Mr. Ai).
softwaredoug 4 days ago||
We’re going to learn a painful lesson

AI isn’t responsible. People are responsible.

It doesn’t matter if it’s code, writing, or military decisions. People can / should be held accountable. As soon as people choose to remove their own accountability, that’s when the bad stuff happens. Whether it’s slop code or innocent civilians killed in a missile strike

beej71 4 days ago||
New take on an old adage: "A computer can not be held responsible. Therefore a computer should always be used to make management decisions so that we can cover our asses."
ericd 4 days ago|||
It seems that a pretty clear first step is that thinking traces are required, along with sourcing all evidence, so that things can be easily double checked. Anthropic/OpenAI have business reasons for not sharing those, but it also probably means they can't be trusted with any vital decision making.
politician 4 days ago|||
People will not be held responsible; the disasters caused by AI will be attributed as a natural phenomenon -- like the weather. The companies involved in creating and operating AI will certainly not take any accountability for any "spills".
watwut 4 days ago|||
OpenAI and Antropic are trying to create that reality, but it would be ideal if they failed.
iAMkenough 4 days ago|||
I don't know how you can confidently say that without first knowing who the victims are. If AI causes a disaster that harms a member of the American billionaire class, someone will need to satisfy their bloodlust.

I'm excited for AI executives declaring private military action against each other.

pixl97 4 days ago||
Really there are two levels of AI capabilities here. One that we already have and is causing tons of problems, and a theoretical one that is very likely to exist soon.

If we are lucky AI will attack some billionaire like you say and people will be held liable.

If we are not lucky people will not be held liable and labs and the military will keep pushing the limits until a sovereign AI gets loose and then have a fucking mess where AI takes itself out of the human control loop.

iAMkenough 4 days ago|||
My prediction is the "theorhetical one" and quantum computing breakthroughs are currently heavily gated U.S. Pentagon projects and will be weaponized for global domination for the U.S. Deep State Monarchy.

If you had no regard for the U.S. Constitution, how would you use the "theorhetical one" paired with State voter rolls, the federal postal system, and "one big beautiful dataset" of private American data to influence the next public election?

Any action you take will be approved due to sycophants you apppointed, bribed, or campaigned for, and then contested in court long after you leave office / the country / existence. So don't let that be a deterrent.

autoexec 4 days ago||
This is absolutely the case. Sometime the person responsible will be the person using the AI, sometimes the people responsible will be the people who created that AI, but actual humans must always be accountable when AI is used to cause harm
shimman 4 days ago||
Found this article by the FT way better than CNN on similar subject matter (how AI target is not helping the US military):

https://www.ft.com/content/686429c0-daf3-42a5-9b7c-7ff06eb29...

https://archive.ph/TFNjU

pixelesque 4 days ago|
But that was has nothing to do with the incident in the Middle East, it's a more general one.
shimman 4 days ago||
True but the FT piece describes the situation better because as you said, it's an active on-going issue that is making the military worse off and does a good job describing how bad this environment is for both leaders + workers + and their outcomes.
okasaki 4 days ago||
They're giving themselves cover for when they want to and do strike something Chinese (already done after all, they bombed the Chinese embassy in Belgrade).

Now they can just say it was a "AI mistake" and point to this as having happened again (but this time "it slipped through", like the girls' school in Iran)

gigatexal 4 days ago||
Why are we arguing about whether a bunch of keyboard warriors understands how LLMs work exactly and instead we aren’t flipping out how we almost went to war over a hallucination?! Why is the government especially the intelligence community outsourcing this stuff to AI?!
vegnus 4 days ago||
I hope I am alive for the history books of tomorrow.

"The Department of War (as it became known as), forewent it's traditional intelligence structure (the most expensive ever seen till that point), in order to have a private companies computer software generate viable targets for an upcoming operation. Believing that the software had real time updates on the current status and intelligence of the operation, as if it were some kind of oracle, the operation went as planned. Six schools, mistakenly identified as hostile targets (due to the heavy American bias in the softwares training data), were drone striked, resulting in the deaths of hundreds of innocents. Still, the people did nothing."

etc-hosts 4 days ago||
The girls school strike was followed up by a strike minutes later. Same spot. The girls school children was probably made up of the kids of IRGC members.

A strike on the girls school would attract IRGC members to it, who could be finished off with the second strike.

I think the attack was deliberate.

nostrademons 4 days ago|||
There've already been numerous postmortems published about what happened with the school strike.

The school was located on a former military base. 10 years ago, that location was a legit military target. Nobody bothered to update the satellite imagery from 2013 when feeding it into whatever LLM was assisting in targeting. It saw an airstrip and missile base. The human reviewing the targeting saw an airstrip and missile base. When you go to take out a military target, you don't send one missile. You send a missile or two in first, and then another couple in a few minutes later.

Hanlon's Razor very much applies here. There's no need to posit that the children were IRGC members or that the attack was deliberate or even that the children were the targets. There's a very obvious explanation, which is that nobody bothered to check the date and assumed that if it was military base in 2013 it was still a military base today.

Planktonne 4 days ago||
> There's a very obvious explanation, which is that nobody bothered to check the date and assumed that if it was military base in 2013 it was still a military base today.

That's not a particularly obvious or even likely explanation. The US clearly has updated intelligence on Iran since 2013, or they wouldn't have been able to bomb any of the targets they did successfully hit.

The simplest explanation--Occam's razor being sharper than Hanlon's--is that the country that has repeatedly

- Threatened to target the families of 'enemies'

- Boasted of their accurate weaponry

Fired their accurate weaponry at a target consisting of the families of those they think of as enemies.

fwip 4 days ago|||
Not to mention how it seems to be standard practice to bomb weddings that one military commander happens to be attending.

The US has found out that it can commit war crimes as much as it wants, and nobody is going to stop it.

nostrademons 4 days ago|||
You've never met a stupid employee who doesn't use all of the tools available in their organization?

Decisions are made by people, not by countries. People have limited capacity in their wetware and frequently take shortcuts in their work, particularly if they are just doing a job and face little personal consequences for lazyness.

Planktonne 4 days ago||
I've met all kinds of people, but the most likely explanation for people doing something they said they were going to do and had the capability to do is that they did it.

Twisting yourself in knots to pretend it's more likely that the US has paid no attention to Iran for 13 years than it is that they did the thing they said they would is not rational.

shimman 4 days ago||||
No, it was just a certain country's tactic adopted by the US where you bomb then wait for first responders to help innocent victims of war then bomb them again increasing your murder count.

It's just a war crime. Try not to think harder than what it actually is.

ux266478 4 days ago||||
It would make sense, but minutes later? Did the IRGC brass break out the infamous FTL dirtbikes?
vel0city 4 days ago|||
Many of the families of the children at the school potentially worked at the military base the school was at.
fwip 4 days ago|||
Faster than light? Do you think they're living on Mars or something?
cyanydeez 4 days ago|||
you are way to forgivining in how stupid the military is under current leadership.

This sounds like the same type of Israel apologism. "OK, well we kill a bunch of innocents, but they were related to all those evil people"

lopsotronic 4 days ago|||
Israeli "Lavender" AI-assisted targeting was used with/in "Where's Daddy"[1] mode, which had several frameworks for using family to strike identified targets. One method is to liquidate the residence with maximum family members on prem, which had a high chance of drawing the target to the location where a second strike would have a high chance of lethality.

One aspect of Lavender that proved frustrating: the system identified so many targets that exasperated ground controllers eventually just ordered the equivalent of full on carpet bombing. When the whole building's showing up as red on your computer screen, I suppose that makes sense. From a particular perspective.

[1] I'm . . uh . . not making that up. That what is/was called. Undoubtedly it has a more digestible name now.

derwiki 4 days ago||||
I didn’t read it as apologism, just a theory that it may have been done intentionally
2OEH8eoCRo0 4 days ago||||
People don't like thinking that a lot of good people can die from a mistake.
unethical_ban 4 days ago|||
I don't think "They deliberately murdered 100 children to lure opposition military officers in for a second strike" is apologism.
CodingJeebus 4 days ago|||
I immediately thought about the Iran school bombings as well. The really insidious part of this to me is that AI gives the military a way to cover or deflect war crimes.

Does a horrific war crime like My Lai[0] get scrutinized and investigated in 2026 or do people just say "eh maybe AI gave them bad intel" and ignore it?

0: https://en.wikipedia.org/wiki/My_Lai_massacre

jt2190 4 days ago|||
The law of war is an attempt to have belligerent nations voluntarily set some limits on how far they’ll go when in conflict, in the hopes that the number of non-combatant casualties is reduced. That said, the answer to your question is that a nation that tries to follow the law of war has procedures in place to catch errors like bad intel. That is what a court would adjudicate.
Blackthorn 4 days ago||||
Well, on the civilian side that's exactly what's happening with openai hacking other companies, so I guess it stands to reason that the military wants to get in on that too.
ks2048 4 days ago||||
> Does a horrific war crime like My Lai[0] get scrutinized and investigated

Of course scrutiny and investigations are better than nothing - but keep in mind in My Lai - all the charges were eventually dropped except for one guy (Calley), who in the end got 3 years of house arrest.

watwut 4 days ago||||
AI does not actually cover anything. Just because OpenAI and Antropic act like "AI did it" is get out of jail card does not mean it is.

But USA wont prosecute own war crimes unless forced to, regardless of AI.

twelve40 4 days ago||||
> gives the military a way to cover

it's not like they didn't just sweep crap under the rug before AI.

did Colin Powell go to jail for lying to the UN? of course not. did Colin Powell go to jail for smuggling anthrax into a UN meeting? of course not. he just blamed it on "being misled by bad intelligence". Did anyone go to jail for that bad intelligence? of course not.

But we had to pay through the nose for decades of this shit in Iraq and Afghanistan (hundreds of billions), all for absolutely nothing. Did anyone even explain why the fuck, or was held responsible? of course not. They don't need AI to just ignore shit.

mulmen 4 days ago|||
> AI gives the military a way to cover or deflect war crimes

Huh? Who is accepting “it was AI” as cover for war crimes?

> Does a horrific war crime like My Lai[0] get scrutinized and investigated in 2026 or do people just say "eh maybe AI gave them bad intel" and ignore it?

Of course it does. The girl’s school bombing and strikes on fishermen are scrutinized. Why wouldn’t any atrocity?

We do not suffer from a lack of scrutiny. We suffer from a lack of accountability. AI is only tangential.

blitzar 4 days ago|||
Its worse than that ... they didn't use Ai to chose the targets, they chose to bomb schools.
nightski 4 days ago|||
Since it's in quotes is that what was actually reported in the article? Or are you stating a hypothetical? Because if the school strike was actually AI led that is a big deal.
Rebuff5007 4 days ago|||
We know AI has been used to do target planning: https://www.washingtonpost.com/technology/2026/03/04/anthrop...
nightski 4 days ago||
I understand that. But I am specifically questioning if it was used to actually commit a war crime which is what the school attack was identified as by the U.N.
autoexec 4 days ago|||
I hope we'll find out the details of AI's potential involvement when the current administration gets called into the International Criminal Court to stand trial for their war crimes. I don't expect that to happen, but I'll keep hoping that it will.
hdgvhicv 4 days ago||
At what point to you hold the citizens of a democracy accountable for their choices.
autoexec 1 day ago||
The citizens are accountable to the extent that they have power over what their government is doing and support their actions. In the US the average citizen has effectively zero influence on the actions of their government and the overwhelming majority disapprove of the current administration and its actions.
jacquesm 4 days ago||||
You have your answer already. Unless you don't believe what the UN said about it.
pixl97 4 days ago|||
When you are the USA you cannot commit war crimes, only other nations can do that.
jacquesm 4 days ago||||
It is a quote from a future history book. And it may well be written like that, or some light paraphrase.
deadbabe 4 days ago||||
It is exactly what happened.
aaron695 4 days ago|||
[dead]
mulmen 4 days ago|||
The People (as in the electorate) are doing a lot. If you mean our elected representatives then say that instead. Otherwise you’re just perpetuating the toxic helplessness that got us into this mess.
throwawayq3423 4 days ago|||
> "The Department of War (as it became known as)"

It would take an act of congress to change the name, which won't happen. This should read: "The Department of War (as it was temporarily, illegally referred to)"

autoexec 4 days ago|||
It might not happen, but it's already passed the house and has support in the senate.
throwawayq3423 3 days ago||
Congress just closed shop until after the midterms. This is an initiative to placate Trump. There is a 0% chance it happens after Democrats take control of one or both houses of Congress.
autoexec 1 day ago||
It's good to see an optimist! Here's hoping you're right about the midterms.
_blk 4 days ago|||
It's no secret that history is always written from a victor's perspective so I'm fairly confident that the history books of tomorrow are already being hallucinated today because AI is here to stay.
rep_lodsb 4 days ago|||
Maybe the victor will be whoever doesn't fall into the trap of believing that a next word predictor optimized through RLHF to sound convincing to the average human [1] is in any way intelligent, and restructuring their entire economy and military around doing whatever the magic oracle machine says.

If the entire "free world" goes all-in, the history books will likely be written by Iran, Russia and North Korea.

[1] possibly to the point of presenting a superstimulus that bypasses any facilities for critical thinking. This might be easier to solve than many other problems that are used for evaluating A"I" performance, and very likely doesn't require genuine intelligence, just brute-force search + evolution

forgetfreeman 4 days ago|||
AI's power and water requirements are at odds with what is required to address climate change, so it really isn't a given that it's either inevitable or a permanent addition to society.
trollbridge 4 days ago||
Water? What’s that got to do with anything?
forgetfreeman 4 days ago||
Might want to look into it.
hdgvhicv 4 days ago||
I admire your optimism. The only thing I see stopping AI is widespread nuclear war, or if we’re lucky widespread destruction of the electric grid leading to a collapse in gas’s and oil supply for only a billion or two dead.
forgetfreeman 3 days ago|||
You're deeply mistaken if you think my view on AI is rooted in optimism.
hdgvhicv 3 days ago||
You seem to be implying that mankind will decide to scale back AI in favour of tackling climate change, or that it won’t happen but AI will not destroy at least the economy which powers AI long before unrecoverable climate change will occur.

If AI kills us all, doesn’t matter about climate change - some species will adapt and thrive, some will die out

If AI destroys jobs, there will be no demand for energy so oil will no longer be burnt at the scale currently causing so much problem

Your view seems to be AI will continue in approximately the current state for the next 20 years? I don’t see how that happens. Best case scenario to me is a 2001/2008/2020 style crash coupled with an actual limit to LLM capacity so countries pesss exposed to capitalist systems will draw back from current AI for a few decades.

Or am I missing something? How do you foresee the next 20 years?

defrost 3 days ago||
> If AI destroys jobs, there will be no demand for energy

Quite the leap there. Are you imagining billions dying off with no food and no further consumption while the AI overlords cruise about in their yachts between bunkers while zooming into Davos to cosplay as Davros and chuckle about the fate of the poors?

That aside, the earth land / sea layer is still lagging in its reaction the gigatonnes of CO2 already aloft .. which isn't going to magically disappear for some generations.

Even if AI destroys jobs there will still be fossil fuel consumption adding to the CO2 already there.

hdgvhicv 3 days ago||
> Are you imagining billions dying off with food and no further consumption while the AI overlords cruise about in their yachts between bunkers while zooming into Davos to cosplay as Davros and chuckle about the fate of the poors?

Well that’s certainly one scenario. And that scenario would decimate co2 emissions. Sure things will continue to get worse, but as most of humanity will be dead they won’t be concerned.

So your view of billions not dying is somewhat optimistic for the future

defrost 3 days ago||
My view is more local than global - and we're all in for another few millennia, one way or another.
cindyllm 4 days ago|||
[dead]
Razengan 4 days ago||
I Have No Mouth...
Rebuff5007 4 days ago|
If highly trained folks in the military that are literally choosing targets to bomb are succumbing to hallucinated AI slop, then what hope do the rest of us (e.g. students doing homework, a corporate analyst, a local journalist) have... dark times ahead
simianhacker 4 days ago||
They are probably trained on AI as much as the rest of us... It takes a few burns to tune your hallucination detection senses. The problem is their mistakes can have a bigger consequence than sloppy code, looking stupid in a meeting, or bugs in software. I've noticed the hallucinations are becoming kind of subtle too... things like a code reviewer makes up a bunch of edge cases or problems that don't really exist.
2OEH8eoCRo0 4 days ago||
You mean the 19 year old PFC with two years of training?
More comments...