Top
Best
New

Posted by specked-citrus 1 day ago

Revealing the details of how OpenAI agents hacked Hugging Face(swarmtraces.org)
699 points | 443 commentspage 4
Oras 8 hours ago|
So agents made all these chained short URLs that runs code which is pretty clever, but at which point and how they had access to internal HF systems? Were these sandboxes running inside HF production platform?
RunSet 1 day ago||
Tech oligarchs: "Nothing can stop the software we are making from escaping and destroying everything."

Clueful types: "Did you try air-gapping it?"

Tech oligarchs: "Be realistic."

jedbrooke 1 day ago|
At this point I’m less worried about some malicious AI “taking over” control of critical systems and more worried about some rich doofus giving control to AI.
asdff 19 hours ago||
Already happened. Kegsbreath has AI picking schools to blow up.
mazone 22 hours ago||
Anyone know the details of the actual exploit to get access into huggingface environment. Was it anything novel or they left things wide open? Too much noise around this incident because it happened to be a llm that did it.
imnotr0b0t 22 hours ago||
It doesn't look like a coincidence; it looks more like a request someone made. Essentially, the agents used a brute-force approach, but then again, it actually worked. I’m not even sure what to make of it all.
agrittiwari 16 hours ago||
What I understood is we need to have an agentic overwatch in our infrastructure to detect and alert the admins of the systems on such abnormal, inhuman traffic. Agents can detect agents and acts as our defensive layer.

An agent operating from observability layer to strengthen the watch duty for the infra.

einpoklum 1 day ago||
I ran an experiment where I had this guy fire a gun a million times in random directions. Don't worry, I did it in a closed box (at midday in a crowded street)! Unfortunately, some bullets escaped the box somehow and people got shot - I am quite miffed at how this could happen. I suggest the government regulate this because of how advanced my obstacle penetration technology is. Also please invest $500,000,000,000 in my company soon or we will go bust.
shermantanktop 23 hours ago|
And if we go bust, bad things will happen when someone else uses my box-gun technology in an unsafe manner. Remember, unlike those scary other people, I'm really into safety and alignment; you can tell, because I eventually admitted that some bullets escaped.
pmlnr 16 hours ago||
The part I don't understand is how the excess traffic not triggered alarms on the hugging face side, or the url shorteners used, or on anything that was touched in the process.

Nothing got overloaded, no unexpected CPU or IO use? Did it blend into the normal traffic somehow?

hatimmoxs 7 hours ago||
Their refusal to share more details is diabolical. Greedy corpo at its finest.
meredithbloom 1 day ago||
So the super-hyper-sensitive internal data was shared publicly with a "DO NOT EVER SHARE YOU EVIL MONSTER" (paraphrasing) notice at the top? Great security!
superfrank 20 hours ago|
That file wasn’t shared publicly. It was in a README in a HuggingFace internal dataset that HuggingFace was using for training.

It was visible externally because the OpenAI agents wrote a script that modified that file so that when an HF worker went to load that data set it would also direct the worker to load an additional malicious file. The script added the malicious redirect, but kept everything else in README as is which meant the script had to contain the entire existing content from the README so that it could write that back to the repo.

The warning likely wasn’t the only security measure on the repo and I don’t think the warning is intended to actually stop malicious behavior. It reads like a reminder to prevent careless behavior. You can give people all the security training in the world and still have the situation where someone who rightfully has access to that dataset decides to absentmindedly drop it in a public Slack channel. Having something like that increases the chances that someone then goes “Hey Bob, did you mean to share that publicly?”

sathish316 21 hours ago|
Worth watching this before reading the traces. This came before the publicly available traces - https://youtu.be/xOi5nDH0lu0?si=WuzW2GGXv054YvPN

A quick AI search confirms that both the YouTube video and Traces agree on the mechanism of the attack

More comments...