Posted by artninja1988 13 hours ago
If we consider the amount of RCE/CVE in a software to be limited, I expect these models to result in massively more secured softwares, not less.
>” you only need to find one flaw to exploit a system”
I see this everywhere, especially in these threads and it’s not even remotely true for modern architecture.
Between principles like zero-trust, defense in depth, etc. we’ve been away from the one flaw situation for a long time.
Now does crap software exist that doesn’t follow these principles? Absolutely. But those were a problem before AI.
AI isn’t going to change any of the principles of secure design. It’s just going to punish those who aren’t following them.
I disagree with your take that "it's not even remotely true" and "we've been away from...". We really really haven't. This is as true as it has always been. Any system is as secure as the weakest link. That link can be anything from a human, to a leaked token, to a badly configured server, to bad code running somewhere. The amount of leaks / ransomware attacks / etc in the past 5-10 years serve as ample evidence.
And now, right now, there are "red team" capabilities that can literally bang tokens against the wall until they find that weakest link, and then can move laterally with inhuman speed. That's the reality, now. The "blue team" capabilities are lacking, because the bottleneck is with humans. From alert fatigue, to not enough trained people, to having to vet every new RCE, to having to test, deploy and validate any mitigations, the scales are currently favouring the automated side.
I think "everyone" is doing heavy lifting here. It's not clear to me at all that a powerful model released with no restrictions would be a net positive. This hinges on the hope that the under paid, under motivated, under staffed and under qualified security teams at many random corps are going to leverage those open models to fix their vulns faster (and better), than highly motivated attackers will use them for offense. I'm not super confident on that.
It's gotten to the point now where we literally have the frontier labs saying, "hey, so we created this AI which presents biological, chemical and cybersecurity threats to the public, oh and it also has self-improvement potential. We tested it to see how crazy this thing is, and it was a total shit show, breaking out of our sandbox then proceeding to hack a bunch of stuff. But don't worry we're taking this very seriously – we're going to continue to development and test, but try a bit harder to cage it going forward".
It's honestly absurd just how predictable all of this is to anyone who frequents AI doomer communities...
The idea that you can cage an AI which is breaking leet coding records is so dumb it's hard for me to even have theory of mind for the people who think this is reasonable. And the big brains who think this are genuinely arguing crap like, well we'll just use the AI to patch the problems with our cage.
But there more!
AI optimists used to argue that we'd never be so stupid to hook up advanced AIs to the internet. Lmfao!!
AI optimists used to argue that we'd obviously not be so stupid to create an AI whose sole goal is to maximise the number of paperclips in the universe. And I guess we haven't built that, but it's not because we're not stupid enough to do it, but just that we'd prefer to create AIs whose sole goal is to maximise the number of offensive cybersecurity challenges it can beat.
I think the whole way we doomers have been way too charitable. We always assumed that people will care about AI risks, and try their best to mitigate bad things happening. That bad things would happen by mistake. We never even bothered modelling the scenario where people would just simply not care, and even as the AI we all warned about was being created invent conspiracy theories on internet forums about how bad things aren't really happening and it's all just a marketing gimmick.
I hate ranting like this... I'm sorry for not picking my words more carefully. I'm just getting so angry and fed up with this. This is my life and my families life on the line. I don't care about the economic potential of AI. I just want myself those I love to have the chance to live a normal life without having to be worried about what some moronically unserious AI company is building next.
A year ago I was felt like there was at least possibility people would see the warning shots and try to get us back on the right path. But this just isn't happening...
> "ai model leaks from openai and attacks huggingface"
to be somehow framed as
> "and therefore openai cannot be trusted with ai safety, and we need open weights models".
anybody have an idea how to make this easily digestable?
- sharing the model with DoD, NSA and Israeli government
We are sharing this because we believe it’s important to be transparent with the public and the safety and security communities about this potential shift in capabilities.
Yet another PR piece. Sigh.