Posted by rudy6912 11 hours ago
Our model hacked some website
Our car ran over some peopleExtrapolating this, we should expect this kind of breach to occur more often. Humans are simply not capable of contemplating every fail scenario for swarms of thousands of intelligent autonomous agents which can seamlessly and instantly share knowledge. We need independent audit and monitoring systems to assess the intent of each task and align it - in real time. This is far harder than it may first appear.
There is also a broader discussion about social utility. Cars are fantastic, but 37,000 people die every year from car accidents. We accept that there is no way to make cars perfectly safe, so we accept the cost relative to the benefits. I think we might have to make a similar bargain with AI. The problem is that the potential costs are far higher with AI, and they're not easy to predict.
I may be too close to the research, but it appears to me to be so hard as to be unrealistic.
I recall some story a while back where an auditor wanted to see all TCP packets printed out on paper, and it had to be explained to them that this would require a continuous supply of trucks.
Tokens are regularly priced in cents or single digit dollars per million tokens. It's not quite a word per token, but yeah, nobody's reading all that.
Worse, we don't always know the intent even when looking. We have a few tools to attempt it, for example the (misleadingly named) "chain of thought", but that's more like a notepad and the better models get the more they can, for lack of better words, read (and write) between the lines. We have probes and J-space* is the most recent one I'm aware of, but we are still scratching the surface with how reliable and general these are.
But you said "need"; the need for something can be present without that thing being possible.
I also think the intent problem overlaps a frustrating amount with philosophical and political questions. It's the basis for Asimov's Three Laws of Robotics (1942). Intent is subjective. Language is subjective. Humans are imperfect at using language to accurately portray intent. All of these guarantee that an enormous number of queries in the future are going to be misinterpreted. Not such a big deal when it's about a cake recipe, but when it's about governance, laws, military targets, nuclear power sites, etc, the scope for failure becomes catastrophic. The Three Laws of Robotics attempt to create a backstop, but as countless stories have explored since (including I, Robot), even these laws are subject to interpretation.
We are giving computers human perspective intelligence but they are not humans and hence do not have the same shared assumptions.
While I wouldn't put it past e.g. Musk or Zuckerberg to think themselves above such outcomes, and I trust the people who keep telling me Altman is just as bad, this is a really really terrible idea if he is doing that for something as mundane as a regulatory capture.
* depending on the details of the hack; this doesn't look like it would be that, but given they shouldn't have done this at all, there's no reason to predict a specific level of maximum damage before being caught, and hence no reason to predict a specific threshold for government response.
Sounds like a great concept for the next season of Danger 5.
That said, I can hear the accent in my head:
The name's Bond. Bruce bloody Bond.
"Cyber experts believe these systems were poorly protected - but that's not the point" is the actual subheading.
Yes, it's the point. Lots of people have been rightly saying all this stuff was inadequately secured for many years and this promotion of the idea of perfect security being even possible is a major problem.
The real story here, unsurprisingly, is government website was poorly operated and got hacked.
"This was just the latest case of AI agents ignoring laws around how to safely access online information and perhaps the most serious yet given the information was government controlled."
So who was actually running the agent? Was it sandboxed? These people, and many apparently in these labs, that believe if you just tell the agent in English what the rules are then it should follow them are at best utterly naive, and at worst deliberately dangerous.
But the fact these governments making the noise are the ones promoting mandating everyone giving up their information to be then stored so incompetently, while they point fingers around at everyone else is just beautiful. And then deploying midwit armies to tell people what to think about it . . . the AI takeover can't happen soon enough.
If you can’t stop openai from accidentally, or at least non-maliciously, accessing my private info.. then you definitely can’t stop the bad guys!
The point is moot anyways. All info about everyone is out there for the taking now by a half-competent AI prompter. What do we do about that is the real question?
It's like filing your gov tax return in a five eyes country: you guys actually know the answer already, just save me the trouble. But we have to act like they haven't been spying the whole time.
If we actually distributed the benefits of mass surveillance and privacy invasion (and now add wilful copyright infringement) then it would be enormously less objectionable.
Come on. If "AI Agents" (scare quotes) could do it then in practice anyone could have been doing this the whole time and no one would have known.
The same principal applies to government sites. If something is poorly secured and adversaries are using it to steal stuff then there are avenues to report it and get it fixed up.
>But the fact these governments making the noise are the ones promoting mandating everyone giving up their information to be then stored so incompetently,
The governments in these cases are bought by the very people you're defending. They're using it as a convenient proxy because they know people like you won't look behind the curtain and see corporations pushing this shit so that they can steal freely and just point elsewhere.
Imagine if I committed a murder, and then say it was my guy who did it acting rogue.
It is time for these companies to start being responsible for all the shit they are doing.
That employee might or might NOT be found guilty too, possibly to a different degree, depending on the circumstances.