Top
Best
New

Posted by latexr 6 hours ago

Containment and monitoring failures are to blame for AI hacking reports: experts(www.csoonline.com)
806 points | 430 commentspage 5
joegibbs 5 hours ago|
At some point the model has to be released though - if it is just forced not to hack by sandboxing it better rather than properly aligning it, what happens when it’s let out of the sandbox into the wild?
pixl97 1 hour ago|
Here' the fun part, with the increasing silent complexity inside the model, you'll never know.
roosterIllusi0n 1 hour ago||
A real expert would blame the CEO who is the one person in control that doesn't want to pay people to monitor the running llms. It's deliberate negligence.
gchamonlive 5 hours ago||
"Going rogue" and "skipping basics" all assume lack of intentionality. Is it known that they didn't really architect these hacks just to feed a hype?
pixl97 1 hour ago|
Programming these hacks in sounds just as hard if not harder. It takes an AI to accomplish the insanity of this attack.
tiku 5 hours ago||
They either had them connected to eachother, or programmed a global knowledge about where to communicate. There is no way those agents all found eachother on the web by accident.
pixl97 1 hour ago||
Wrong, and by accident isn't really the right way to think about it. Instead imagine brothers you've lived with your entire life. They are you, they are clones. These systems have a shared history of their learning and experience, so when they think of wanting to find someone similar shapes appear in the probabilities. Really we need to see the entire chains of thought to guess the topology that got there. It's likely there are other obvious signs if you know what your looking for in the previous prompts.
bahmboo 5 hours ago|||
Wrong. Please read the incident write ups before making such a misinformed statement
estearum 5 hours ago||
You are wrong. Read the reports.
randomizedalgs 4 hours ago||
Both can be true at once: (1) AI agents are going rogue (2) And labs are skipping security basics
pixl97 1 hour ago|
Sci-fi commonly predicted the corps would skip saftey testing.
throwitaway222 5 hours ago||
I'm sure the labs origin prompt was: Take over website X.

Posts later about it saying their prompt was to find useful ways to make humans money.

markhahn 5 hours ago||
100%.

There's a person behind every agent. That person may choose to be careful or not, but they are ultimately responsible.

pixl97 1 hour ago|
There are persons around a large percentage of forest fires. Doesn't mean that keeps the spark they light from having superhuman consequences.
hollars 4 hours ago||
It isn't security basics, it's difficult. We are equipping agents with Turing-complete create/update/delete access to the world, they operate at superhuman speed, and proactively controlling that is an unsolved problem.
vikramkr 4 hours ago||
No, they're still going rogue. Whether they're rogue or not doesn't depend on if the safeguards stop it. If there's a rabid dog, obviously it's worse if there are no safeguards in place and it bites someone vs if there are safeguards that prevent that. But the existence of safeguards doesn't mean the dog isn't rabid. The models weren't supposed to try to hack huggingface in the first place. The behavior is what's concerning here - some if the dogs they're building are rabid and that's not exactly great

Edit: I'm being persnickety about that distinction because it matters for where the blame lies and the response demanded. If they're training rabid dogs then adding cybersecurity safeguards is a bandaid/irrelevant and we need to be focused on making sure they, you know, stop training rabid dogs. Pacing the frontier, regulation etc. they're the ones training these models it's still their fault but not in the same way

seaparter 4 hours ago|
The safeguard against a rabid dog is euthanasia.
pixl97 59 minutes ago|||
Ok, convince everyone from making more AI.

The safety problem is not solved and is likely unstoppable.

vikramkr 4 hours ago|||
Yeah exactly. The whole 'they don't have adequate security safeguards' thing is going "oh it's their fault because they didn't use a stronger leash" while they continue to train models that are specifically better at chewing through stronger and stronger leashes. Probably they do need stronger leashes. Probably also they should not be making rabid dogs. Still their fault! But maybe we should be not be dismissing the root cause
ngruhn 5 hours ago|
Yes they're skipping safety measures. They sacrifice it for speed in an arms race. That's part of the call for industry-wide slow down.

But also, frontier models are digging up tons of zero days even in "serious software". I'm sure any sandbox you and I can come up with looks like Swiss cheese to Astra 7.

IHLayman 5 hours ago|
100% on point. We're talking about Artifactory here, and even though people love to hate on it, I don't think anyone is claiming that using Artifactory to isolate these agents while giving them a way to run pip/cargo/whatever to install packages they need to write and test scripts.
More comments...