Posted by artninja1988 12 hours ago
OpenAI messed up and they are saying they will pause so they can do better.
They are not saying that other orgs who may already be doing better should pause.
Just like Reddit you come here for clickbait outrage, not level headed analysis.
I don't think there's too many people who distrust AI companies but trust the current government or the CIA as impartial authorities. I'm not saying you don't have an argument, but appeals to that specific authority will not be effective except among people who already agree with you.
I dont worry about AGI newrly as much as about Thiel, Karp, Musk, Ellison, Zuckenberg, Trump, Vance, Rubio, Miller and the rest of them.
When OAI demonstrates these dangerous capabilities live in a public environment where security experts can see and verify what actually happened, then reasonable people can have reasonable discussions about the level of danger.
This is a very low evidence bar.
Right now you are running in circles yelling "the sky(net) is falling" based on details sourced entirely from OAI. Oh yeah, no way a trustworthy company like OAI would ever bend the truth to serve their own purposes.
Open models are on their heels and their attempts at regulatory capture are not moving as fast as they would like. So it's time to market this incident in a way that gives them monopoly on closed models, with heavy safeguards that are only lifted for selected customers, and laws limiting the use of open weight models.
If we consider the amount of RCE/CVE in a software to be limited, I expect these models to result in massively more secured softwares, not less.
>” you only need to find one flaw to exploit a system”
I see this everywhere, especially in these threads and it’s not even remotely true for modern architecture.
Between principles like zero-trust, defense in depth, etc. we’ve been away from the one flaw situation for a long time.
Now does crap software exist that doesn’t follow these principles? Absolutely. But those were a problem before AI.
AI isn’t going to change any of the principles of secure design. It’s just going to punish those who aren’t following them.
I disagree with your take that "it's not even remotely true" and "we've been away from...". We really really haven't. This is as true as it has always been. Any system is as secure as the weakest link. That link can be anything from a human, to a leaked token, to a badly configured server, to bad code running somewhere. The amount of leaks / ransomware attacks / etc in the past 5-10 years serve as ample evidence.
And now, right now, there are "red team" capabilities that can literally bang tokens against the wall until they find that weakest link, and then can move laterally with inhuman speed. That's the reality, now. The "blue team" capabilities are lacking, because the bottleneck is with humans. From alert fatigue, to not enough trained people, to having to vet every new RCE, to having to test, deploy and validate any mitigations, the scales are currently favouring the automated side.