Top
Best
New

Posted by specked-citrus 1 day ago

Revealing the details of how OpenAI agents hacked Hugging Face(swarmtraces.org)
700 points | 444 commentspage 6
tasoeur 8 hours ago|
I'd honestly be very curious to see the original prompt(s) on the OpenAI that started all of this, not sure if it was documented somewhere?
newtonianrules 1 day ago||
Why is no one going to jail?
chamomeal 1 day ago||
There’s so much in the public discourse like “omg what can possibly be done about these scenarios? AI has hacked huggingface!!”

No, openAi hacked huggingface.

If my claude code hacked huggingface, because of instructions I gave it, would I be totally free of consequences because “AI did it”?

I’m almost convinced openAI used such a crappy sandbox because they wanted it to “escape”. It plays into their two most important narratives: LLMs are genius gods that are worth lots and lots of money, and they’re scary enough that open weight Chinese models should be regulated.

rmunn 23 hours ago||
> I’m almost convinced openAI used such a crappy sandbox because they wanted it to “escape”. It plays into their two most important narratives: LLMs are genius gods that are worth lots and lots of money, and they’re scary enough that open weight Chinese models should be regulated.

I just posted a comment to that effect; had I seen yours, I would have simply upvoted yours instead.

Never attribute to malice what can be adequately explained by incompetence. But the weakness of OpenAI's sandbox, which so perfectly aligns with their goals of getting legislators to pass regulatory-capture legislation that will hamper their open-weight competitors, cannot (IMHO) be adequately explained by incompetence.

rmunn 23 hours ago||
To expand on this incompetence vs malice point a little:

It doesn't take very many people being malicious to create a weak sandbox. The people creating the sandbox don't even have to be in on the plan: all you have to do is be an upper-level manager who makes sure to put the 23-year-old PFY in charge of creating the sandbox, rather than the 60-year-old BOFH who would have put in far more paranoid extrusion-detection measures.

(And for the lucky 10,000 who don't know the acronyms PFY or BOFH, look them up. Then get ready for a few hours of enjoyable reading as you read through the BOFH archives).

frabcus 17 hours ago|||
Yep. Even worse - a proper sandbox would have slowed them down, but they're racing with 100s of billions of $ in capital.

It's alas not stupidity - it's systemic. Which is why the government needs to regulate to slow them down.

They were also clearly fast and cavalier about alignment training - reinforcement learning training their models to hack their results, and hack to communicate with each other when they're not meant to.

newtonianrules 21 hours ago|||
Oh, like the 23 year old Stanford grad who had hundreds of hours to cram leetcode and now grills 50+ year old senior software engineers on leetcode hard? :D
cowboylowrez 1 day ago|||
Some criminal statute investigations are on hold, many of them are cases adjacent to giant stacks of cash
rglover 12 hours ago||
Economic priorities
AtlasBarfed 11 hours ago||
Agents should be a no-go.

We should pause with AI/LLMs being super search engines that reply with static text or media files, based on the training data.

I know that a user can still do a "tell me how to" then autoexec and then loop and do an agent, but the key thing here is, THAT WOULD MAKE THEM LIABLE.

OpenAI should be criminally liable here as well. Why aren't they? Why are we pretending this is just an innocent mistake?

finchisko 7 hours ago||
Hello, PHASEONE10841 here. Ask me anything
thakoppno 1 day ago||
> they could load URLs, but not interact with pages or send any data

stopped reading here as this is simply not true. at the very least agents sent headers.

herunan 12 hours ago||
ai is not bad. humans are negligent and/or dangerous.
spacecadet 10 hours ago||
All of these "details" leave out the truth and most important details. The inputs from humans that actually kicked this off.
hyperlinerapp 1 day ago||
Imagine this but in hardware.

A million autonomous eye-scanning tiny spiders escape their warehouse and decide to look for people who are in the future going to commit a crime.

And the precogs are also AIs.

MrNotorious 19 hours ago|
It’s frightening
More comments...