Google says criminal hackers used AI to find a major software flaw

Posted by donohoe 23 hours ago

200 points | 145 commentspage 3

skeledrew 14 hours ago|

Wild that they think restricting access to models will help much. Access to Chinese models will definitely not be restricted and have enough capability to find exploits as well.

sowbug 15 hours ago||

Security will be a wedge to restrict the sophistication of open-weight and local LLMs, just as it's been used to demonize and restrict cypherpunk technologies.

JumpCrisscross 14 hours ago||

> Security will be a wedge to restrict the sophistication of open-weight and local LLMs, just as it's been used to demonize and restrict cypherpunk technologies

Unlikely in America or China. This is not a game either can singularly control, and locking down the R&D means conceding momentum to the party that doesn't. Which means use restrictions will be contained to countries satisfied with playing second fiddle.

Instead, I suspect we'll see momentum towards running software on publisher-controlled servers so the source code can be secured through obscurity. It isn't perfect. But it might be good enough to get us through this transition.

ls612 14 hours ago||

If America just banned all chinese models that would wipe out most of the open weights landscape in AI, especially anything close to the frontier. I could easily see that happening if a Mythos tier model comes out of a Chinese lab in early 2027. It doesn't meaningfully change the research competition between OAI/Anthropic/Google/SpaceX but it does pad all of their pockets by removing cheap competition and it gives the government far greater control over AI usage de facto.

JumpCrisscross 13 hours ago|||

> I could easily see that happening if a Mythos tier model comes out of a Chinese lab in early 2027

I don't. I'm not saying American politics isn't capable of doing it. But I don't see us being stupid enough to try locking ourselves out of a technology that everyone else has access to.

lazide 9 hours ago|||

Did you not see the foreign drone parts bans?

ls612 13 hours ago|||

But we wouldn’t be. I’m assuming that the US labs retain several months’ lead for at least the next couple of years.

UltraSane 13 hours ago|||

How would it be possible to ban Chinese LLMs?

ls612 12 hours ago||

Place the chinese labs on the entities list. That stops any legitimate company using them and probably makes HF take them down. Sure there will be torrents but the laws for doing business with a sanctioned entity bite much harder than the laws around copyright infringement.

JumpCrisscross 12 hours ago||

> Place the chinese labs on the entities list

Ironically, this–a nascent industry and budding industrial cluster–is the textbook case for deploying tariffs. America tariffs American use of Chinese models and pays that back as a tax credit to American developers.

kshacker 15 hours ago|||

As long as it is within the country, restriction works. How do you restrict the capability from a foreign entity, especially a hostile one?

jazzyjackson 14 hours ago||

netsplit, I guess. decide that the risk of an open network is too great and simply block all routing out of the country through the ISPs and consider the political power that goes along with a global satellite constellation under rule of a single, government-aligned corporation.

notsound 14 hours ago|||

"simply block all routing out of the country" is doing a lot of heavy lifting. For government networks, sure. For civilian networks? It's a bit like stopping pirates from ripping video; how do you deal with an attacker that ultimately can gain some form of access? Even in North Korea external media can be smuggled in.

bluGill 14 hours ago|||

That works for very oppressive countries. However, more freedom-minded countries are not going to law for that.

somewhatgoated 14 hours ago|||

Didnt work out so well with the cypherpunk technology so there is hope

2ndorderthought 14 hours ago||

If they tried to lock down local models more people would use them. They would also have to take down a few us companies in the process who would go down fighting for certain.

xnx 14 hours ago||

Dupe: https://news.ycombinator.com/item?id=48096712

skeledrew 14 hours ago|

This is 3 hours earlier than what you're sharing.

xnx 13 hours ago||

Not sure how article merging goes, but this one shows up as 4 hours later to me.

CrzyLngPwd 15 hours ago||

People used LLMs to find flaws in Google software.

adrianmonk 13 hours ago||

If you're talking about the incident described in the article, it says it was a flaw in "a popular open-source, web-based system administration tool".

Google's blog (https://cloud.google.com/blog/topics/threat-intelligence/ai-...) says Google "worked with the impacted vendor to responsibly disclose this vulnerability", so in this incident, it's not Google software.

amelius 15 hours ago||

But did they use Gemini?

Andrex 14 hours ago|||

> the company added that it did not believe it was its own Gemini chatbot.

-TFA

freedomben 14 hours ago|||

I don't know, but given how often Gemini refuses benign requests IME, I would suspect it's a complete non-starter for finding security holes.

wnc3141 15 hours ago||

But in exchange we get to also waste vast energy and carbon while depleting job prospects for just about any college grad.

andrepd 14 hours ago|

It's not all bad though. We also managed to turn the Information Superhighway of the 1990s into the Slop Wasteland of the 2020s.

plexescor 7 hours ago||

But which AI exactly, theres this new claude Mythos about wihch everone is talking, is it legit or a fluff

kuboble 7 hours ago||

Given how everywhere software is now being written by the LLMs, how is that a top headline news that some (albeit malicious) software is being written with LLM?

The robbers used a CAR in the robbery.

The blackmailer used a TYPEWRITER to write blackmailing letter.

ChrisArchitect 9 hours ago||

Source: https://cloud.google.com/blog/topics/threat-intelligence/ai-... (https://news.ycombinator.com/item?id=48096712)

Why collect all the news dupes but not the source up top OP? Because the source was already submitted?

skywhopper 14 hours ago|

Drives me nuts that the NYT just uncritically cites Anthropic’s unverified claims of “thousands of zero-days” without a hint of skepticism.

More comments...