Posted by latexr 5 hours ago
And I think we generally recognize that rogue AI agents are a form of backdoor bragging -- "our agents are so powerful that they hacked an outside system. Did I mention how powerful our agents are? They are quite powerful. So powerful they hacked an outside system".
But Mistral never hacked anything. Yeah, neither did my toaster.
I could set up a shitty sandbox on my laptop, but I’m pretty sure an agent I place inside of it can’t hack HuggingFace.
Now, OAI deserves a lot of hate. It's more like a nuclear spill when they are refining uranium. You're playing with dangerous things. You have dangerous accidents.
But the opposition does not want to admit how dangerous AI has become and how much more dangerous it's going to get. Not sure if it's religious, or human egocentric, or ignorance of complex systems. They refuse to see the danger actually exists.
That's the thing. These evals were not effective as they failed to evaluate if the model was hacking or not.
Further shy is access to the internet required at all during evaluation? They could run the evals in a fully air-gapped system that simulates the internet. While that's expensive its not unfeasible.
Poor excuse. Internet is prod env, for testing do a proper test env.
Just give the models in training access to a snapshot from archive.org - which frontier labs surely have a crawled copy of already.
The huggingface incident seems to have resulted from them partially mirroring a repository, but for no reason I can see not fully mirroring it, after which they could have unplugged the ethernet cable. This is just stupid.
Yes, they're stupid. Maybe also highly intelligent. But stupid.
Also irresponsible.
They may talk about rogue agents, sandbox escapes, but this will not solve the core issue: secret training recipes are a security hazard in themselves. Full transparency is needed.
You can't make a generalized model without generalized behavior.
Und Sie haben, wie ich sehe, da ein so hübsches, ein so entzückendes junges Mädchen zu nichts andrem zu brauchen gewußt, als daß Sie ihm ein Messer in den Leib gestochen und es kaputt gemacht haben! Halten Sie denn das für richtig?
AI in reality: "I've turned you loose, now make me money"
After all Norton are the experts in preventing computer badness, and OpenAI can't stop blabbing about how much badness their products are causing. A match made in heaven!