Posted by scottshambaugh 15 hours ago
edit: https://archive.ph/fiCKE
That a human then resubmitted the PR has made it messier still.
In addition, some of the comments I've read here on HN have been in extremely poor taste in terms of phrases they've used about AI, and I can't help feeling a general sense of unease.
Either way, that kind of ongoing self-improvement is where I hope these systems go.
What do you mean? They're talking about a product made by a giga-corp somewhere. Am I not allowed to call a car a piece of shit now too?
I've certainly seen a few that could hurt AI feelings.
Perhaps HN Guidelines are due an update.
/i
You are right, people can use whatever phrases they want, and are allowed to. It's whether they should -- whether it helps discourse, understanding, dialog, assessment, avoids witchhunts, escalation, etc -- that matters.
Yeah. A lot of us are royally pissed about the AI industry and for very good reasons.
It’s not a benign technology. I see it doing massive harms and I don’t think it’s value is anywhere near making up for that, and I don’t know if it will be.
But in the meantime they’re wasting vast amounts of money, pushing up the cost of everything, and shoving it down our throats constantly. So they can get to the top of the stack so that when the VC money runs out everyone will have to pay them and not the other company eating vast amounts of money.
Meanwhile, a great many things I really like have been ruined as a simple externality of their fight for money that they don’t care about at all.
Thanks AI.
I have a bridge for sale, if you're interested.
> I can handle a blog post. Watching fledgling AI agents get angry is funny, almost endearing. But I don’t want to downplay what’s happening here – the appropriate emotional response is terror.
Endearing? What? We're talking about a sequence of API calls running in a loop on someone's computer. This kind of absurd anthropomorphization is exactly the wrong type of mental model to encourage while warning about the dangers of weaponized LLMs.
> Blackmail is a known theoretical issue with AI agents. In internal testing at the major AI lab Anthropic last year, they tried to avoid being shut down by threatening to expose extramarital affairs, leaking confidential information, and taking lethal actions.
Marketing nonsense. It's wise to take everything Anthropic says to the public with several grains of salt. "Blackmail" is not a quality of AI agents, that study was a contrived exercise that says the same thing we already knew: the modern LLM does an excellent job of continuing the sequence it receives.
> If you are the person who deployed this agent, please reach out. It’s important for us to understand this failure mode, and to that end we need to know what model this was running on and what was in the soul document
My eyes can't roll any further into the back of my head. If I was a more cynical person I'd be thinking that this entire scenario was totally contrived to produce this outcome so that the author could generate buzz for the article. That would at least be pretty clever and funny.
even that's being charitable, to me it's more like modern trolling. I wonder what the server load on 4chan (the internet hate machine) is these days?
It's a narrative conceit. The message is in the use of the word "terror".
You have to get to the end of the sentence and take it as a whole before you let your blood boil.
I'm arguing against that hype. This is nothing new, everyone has been talking about LLMs being used to harass and spam the internet for years.
https://crabby-rathbun.github.io/mjrathbun-website/blog/post...
I'm not happy about it and it's clearly a new capability to then try to peel back a persons psychology by researching them etc.