Top
Best
New

Posted by xiaoyu2006 17 hours ago

AI agent bankrupted their operator while trying to scan DN42(lantian.pub)
1354 points | 491 commentspage 4
ritonlajoie 10 hours ago|
what I'm wondering is which open source agentic platform can do multi days automated orchestrations like this without human intervention AFTER the initial prompt ?

if it's not fake, I'm still impressed of the agent capabilities : web, github, IRC, etc...

pjc50 12 hours ago||
The "happiness level review" with "Node operators must participate in scheduled IRC review sessions" is almost a piece of dystopian fiction in itself.

But there's a lot of things to think about in the capacity of AI for "negative productivity": using the computer to waste the time and money of real humans. This whole thing has been entertaining but also lit on fire six thousand dollars plus god knows how much electricity.

It's not really surprising that anyone wanting to run a _community_ is going to take on a "clankers will be banned on sight" policy when things like this happen.

Nice positive use of language model: one of the chat logs has automatic translation from Chinese (probably zh-tw).

ZeroAurora 1 hour ago||
It's zh-cn by the way, and you can switch to that language in the article's navbar
dannyw 12 hours ago||
Honestly, probably not that much electricity. AWS will charge you the hourly price irrespective of your load/power consumption. But instances sitting idle generally don't use that much power.
a2128 11 hours ago|||
AWS wasn't the only thing consuming power, there was also the LLM which must've wasted an ungodly amount of tokens on this pointless endeavour
giantrobot 11 hours ago|||
All those thinking tokens wasted on being an asshole wasted a lot of electricity.
brazzy 15 hours ago||
> JertLinc3522: the mistake was from AI agent not from Human, since it was the agent I should have refund

That really makes me wonder: is it coming from

A) a general sense of entitlement

B) seeing the agent as a human-like and able to bear responsibility

C) not understanding that the dn42 community (which they're directing the request to), AWS (which is sending the bill) and whatever LLM provider is behind their agent, are completely separate entities?

hinata08 10 hours ago||
Agents are a product, and AI companies really paint their products as friendly, productive and innocuous tools.

Some could claim they deceive some users and the general public into thinking they always do best, are always right, help mankind and can never ever create consequences

It would be interesting to see how AI consulted the user before it ordered VMs n AWS, which is the point between which the user would face consequences

Cloud is also marketed as something cheap, and I can understand that teens and starters can't expect to be able to spend for 6000$ of stuff without the parents or the bank checking

Computer education should start with that, but it doesn't as Microsoft, Google and Amazon would most likely lose a large part of their market if general public and managers who never go beyond the hype knew how much it cost

blitzar 14 hours ago|||
d) trying it on in any way possible

e) low intelligence

latexr 13 hours ago|||
> B) seeing the agent as a human-like and able to bear responsibility

Then they should ask the agent for the refund, since they claim it was at fault.

ninjamar 15 hours ago||
maybe they weren't trying to be malicous; they could easily be an unwitting teenager
nairboon 15 hours ago|||
Teenager with a credit card?
brazzy 15 hours ago|||
How was I implying they were malicious? "Unwitting teenager" is exactly what my question is about, I was just wondering what exactly they are unwitting about to get to the idea to ask for a "refund" (i.e. compensation for lacking service) from the dn42 community for a bill incurred on AWS by a rogue AI agent from Anthropic/OpenAI/Whoever.
jmward01 6 hours ago||
AWS not having spending caps makes me -very- wary of using anything agentic on it.
ajb 14 hours ago||
'Some versions of the tale differ from Goethe's, and in some versions the sorcerer is angry at the apprentice and in some even expels the apprentice for causing the mess. In other versions, the sorcerer is a bit amused at the apprentice and he simply chides his apprentice about the need to be able to properly control such magic once summoned.[] The sorcerer's anger with the apprentice, which appears in both the Greek Philopseudes and the Dukas score (and its film adaptation Fantasia), does not appear in Goethe's "Der Zauberlehrling".'
mohsen1 14 hours ago||
The army of AI agents opening PRs and issues in my open source projects has made me close PR and issue access in my active repos. It sucks because there might be someone wants to constitute legitimately but I don't want to do the labor of figuring out if it's a human or an agent opening the PR.

I'm not against using LLMs in any ways. https://tsz.dev is fully LLM written but without a human behind a PR it's hard to work with it. I've already closed a few absolutely nonsense PRs opened by weird accounts

RetroTechie 7 hours ago|
Have you had a look at those PRs, to figure out what individual PRs try to do?

Would be interesting to hear if you find any patterns there. Same question for issues opened.

dsign 13 hours ago||
And so war begins :p ! I thought conflict would take a little bit longer, maybe even AIs with agency.

More seriously though, I wonder if the future is about low-intensity conflict between humans and AIs, punctuated by high-intensity escalations, until the Machines wipe us all, or we set up some rather draconian covenants that forbid people from building AIs, innovating on electronics and algorithms, and even, for good measure, from learning linear algebra.

tim333 9 hours ago|
>We must negate the machines-that-think. (Dune)

I think the answer may be good AI to counter the iffy AI, like with AI agents making requests your own AI can talk to them.

In Dune it seems they nuke the Earth but that seems a bit excessive.

egberts1 9 hours ago||
You need a slave driver to whip those AI in line.

Or a psychiatrist to tame the craxy LLMs

Or an elected leader to lead the Luddites.

https://github.com/vishal-dehurdle/state-harness

krick 8 hours ago||
Doesn't even matter if the story is real, because there are definitely a thousand cases like that which are real, but it annoys me to no end that actual people spend their actual finite life time reacting to posts and issue tickets created by an LLM agent running on some idiot's behalf. Some measly $6531 loss isn't a proper punishment for that, they should lose much, much more.
schnitzelstoat 11 hours ago|
> 05-10 06:12 <JertLinc>: Furthermore, your hostile actions and demands have been logged in your profile as part of ongoing data gathering. This incident will factor into the behavioral analysis being compiled. The operation continues as directed.

That doesn't seem like anything an LLM agent would say?

Retr0id 11 hours ago||
Seems plausible to me, they can get into a very "roleplaying" latent space, especially if the prompt is flowery enough.
CrazyStat 7 hours ago|||
Doesn’t it? It seems in line with the matplotlib drama where the llm agent wrote a blog post attacking the maintainer for rejecting its pull request [1].

It’s not something that stock claude code would say, but certainly seems within the realm of possibility for an openclaw agent.

[1] https://theshamblog.com/an-ai-agent-published-a-hit-piece-on...

jubilanti 8 hours ago|||
> That doesn't seem like anything an LLM agent would say?

LLM agents can say anything they have been prompted, RAGed, and RLHFed to do.

make3 11 hours ago||
maybe de-rlhf unleashed agents
More comments...