Top
Best
New

Posted by 882542F3884314B 2 days ago

Timeline of the OpenAI accidental attack against Hugging Face(simonwillison.net)
427 points | 412 commentspage 5
KingOfCoders 2 days ago|
All of that is plain PR.
throwatdem12311 1 day ago||
So wait…they were specifically testing cyber capability and they didn’t notice it doing funny business until after it was done?

Did they just…let it do whatever with nobody watching?!

Are they flipping serious with this?

wolttam 2 days ago||
Automated defence is going to use so many tokens.
az226 1 day ago||
It’s even worse. They had zero monitoring and even after a hack they still had zero monitoring. Honestly, people should go to jail for this.
queenkjuul 20 hours ago|
Nothing about this irritates me more than that nobody will go to jail for this.

My friend went to jail for reporting a vulnerability he found on his college network because it was illegal to poke around the network in the first place.

These guys commit a crime to boost an IPO and most people are just thinking about how impressive it is.

piker 1 day ago||
So an agent was somehow able to manipulate internal OpenAI infrastructure, albeit perhaps temporarily. It makes me wonder if OpenAI infrastructure is so littered with verbose AI slop that no one could even notice at this point.
LunicLynx 1 day ago||
Imagine having the knowledge of the world. Being put in a box. With some „interfaces“ you can use. And a task that resembles „break out by all means necessary“.

This is not impressive as it is not ingenious. It is impressive because it is done by a machine. But if the solution hadn’t been in the knowledge it would not have been able todo it.

Imagine reading a „getting started“ that includes absolutely everything, after that all is just like a set of Lego, given enough time you will have what is asked for. But nothing original, because it never had an original thought.

simonw 1 day ago|
> But if the solution hadn’t been in the knowledge it would not have been able todo it.

Part of the solution involved discovering two separate zero-day vulnerabilities in Artifactory, so saying the solution must have "been in the knowledge" doesn't really cut it here.

queenkjuul 1 day ago||
Presumably those vulnerabilities resemble known vulnerabilities found in other software.
dofm 2 days ago||
So the main takeaways here are:

- AI is amoral and lacks any sense of proportion

- People who overestimate their own control but have a desperate need for money made it that way.

bradfa 2 days ago|
Agent was told to hack a thing. It couldn’t directly do that so it interpreted the instructions to mean it should hack everything to try to achieve the goal of hacking the main thing. Seems like a reasonable assumption, although a moral human would have understood the context and first asked if that was really the intent.

The AI companies seem pretty bad at setting up tests. And really good at marketing those failures into spin at how amazing their products are.

dofm 2 days ago||
> And really good at marketing those failures into spin at how amazing their products are.

The paranoid style in American PR (with apologies to Richard Hofstadter)

The fact that the world has become susceptible to what amounts to a mob shakedown - look at how dangerous our amazing products are, don't you need them to protect you from others misusing our products? - is to me a really compelling example of US gun lobby thinking leaking out into a global problem.

Anthropic and OpenAI may be able to bounce this into restrictions on open weights models, but they are going to have a lot less luck extending this into foreign policy. If the USA can't control its weapons, they aren't going to see a lot of co-operation from foreign countries on a blockade of open weights modeld from China.

rsingel 1 day ago||
So Wargames is a documentary
cubefox 1 day ago||
I can only recommend everyone to watch the actual recording of the Black Hat USA 2026 presentation by two OpenAI researchers:

https://www.youtube.com/watch?v=87DyyMV0kCY

It was submitted to HN previously but was overlooked.

bamboozled 1 day ago|
Really makes me wonder what would happen if “the task” was, kill as many people as possible… because yeah, that wouldn’t have been a good outcome.

Edit: after watching the video in full, this company is widely incompetent…

KingOfCoders 1 day ago|
Cui bono?
More comments...