Posted by takira 10 hours ago
You gave it capability to delete emails. Why did you expect it not to do that at least some of the time? And with enough user some of the time will most likely happen...
Because of the I in AI of course. Would you call it false advertisement and go after the providers?
What does that mean for llms? Their nondeterminism does seem to incline them toward a legal safety requirement. Can you buy a fire extinguisher that 1/1000 times burns your house down? Or can your car brakes instead increase acceleration in rare cases?
Im using llms much more than i used to, but i still cant shake the fundamental stochastic nature of the technology.
Not fundamentally, only until they're compelled to learn from it. The current crop of AI understands neither compelling nor learning.
But wait, hold my beer, now we've got people turning openclaw type tools loose in their systems to do things as sudo or install software packages from supply-chain-attack vulnerable repositories with no human intervention whatsoever!
1) Despite what people say about security and privacy, most are willing sacrifice both for the sake of potential convenience
2) Our priorities for the past decades have been wrong, or the times have changed and we should reevaluate them all
It’s funny that this technology only admits in-band signaling. Given that, any foreign content is risky. It’s actually quite interesting that the current technological ecosystem is built around a high trust situation: npm, pip, cargo all run foreign code in the developer context and communities have norms of downloading random people’s modules.
And so I suppose it’s no surprise that we use LLMs - another tech that is high-trust: since it has no out of band signaling ability.
But it seems like we’re very close to the end of the era where someone will use (in a sensitive system) arbitrary web content carrying the equivalent of merged code/data.
Or will one day some obscure “Unicode homograph” library end up pwning half the world because it was a dependency 10 layers deep for an optional but default-enabled feature that nobody cares about.
Things like Visual Studio’s extension marketplace really acare me. It’s too easy to install Jim Bob’s “starter pack” of extensions that bundles many well known ones with an unheard of one… Or install the wrong “Python” extension because there are 20 with the same icon…