Top
Best
New

Posted by gregnavis 6 hours ago

OpenAI bots knew about the RubyGems caching vulnerability(tenderlovemaking.com)
210 points | 211 commentspage 2
dang 3 hours ago|
Recent and related (others?):

OpenAI agents carried out an undisclosed attack on RubyGems - https://news.ycombinator.com/item?id=49666735 - Sept 2026 (600 comments)

mauriciolange 6 hours ago||
rogue AI agents or AI agents coming from Moulin Rouge?
vidarh 6 hours ago||
Rouge syntax-highlighting rogue agents, clearly.

https://rubygems.org/gems/rouge

Phemist 6 hours ago||
Classic mistake. Tell the agent to highlight this code, but dont give it any actual code. Agent hacks its own gem to find the code to highlight.
codeduck 5 hours ago||
It's carmine all the way down.
goda90 6 hours ago|||
A cabaret AI would certainly be better than one trained on the Khmer Rouge.
PatronBernard 6 hours ago||
At least we know the title wasn't AI-generated?
foobarbecue 6 hours ago||
The weird thing is I've seen LLMs "typo" stuff pretty often. Yesterday I asked Gemini a question about the Python Twisted framework and it answered about Deferreds but misspelled it as "Deferends" in one spot.
bithammerthunde 2 hours ago||
Can we please stop normalizing this behavior. It's not wild it's reckless.

If I let out rats in the canteen, no one is blaming them when people get sick.

There are actual people behind these agents and in previous cases people knew they were "going rogue" and did nothing. This should be reported to the police like any other crime.

sebmellen 6 hours ago||
Did the AI agents actually wear makeup? I’ve never heard of a rouge AI agent :P
shevy-java 1 hour ago||
> In other words, if you publish a gem on RubyGems.org, you can execute arbitrary code on RubyDoc.info.

Well - if rubygems.org could be bothered to fix things, they would not have to rely on rubydoc.info as an external tool. But since rubygems.org sucks (I speak from many years of having used it in the past as developer, until they went loco and added anti-people things such as taking away your ability to remove old gems past a 100k download arbitrary limit), they don't offer documentation. Then again, ruby devs are known to hate documentation. If the ruby core team could only be bothered to fix things, ever since the mass purged other devs ... all coinciding with shopify seizing power. But byroot may disagree on that - after all there is no conflict of interest here. Right?

khalic 6 hours ago||
Oh my favorite typo, you can never go wrong with a little rouge
onlyrealcuzzo 5 hours ago||
I've been wondering if AI will due to programming languages what advanced civilization did to human languages.

It's not just that AI can write Rust as well as Ruby if you ask nicely.

It's also all of these considerations as well.

I hope it doesn't happen, because there's a lot of great languages - I love Ruby so much - but it almost seems inevitable.

This is at the same time everyone and their mother is building their own programming language.

laserbeam 4 hours ago||
There's no such thing as "OpenAI agents" attacked RubyGems. It's someone used agents to attack RubyGems. If they work at OpenAI then it's someone at OpenAI. And if they did it unintentionally, they still did it.

Analogy: if a someone's involved when a person dies, it's manslaughter or murder based on intent. They're different, but they're both crimes.

pavlov 2 hours ago||
“KGB agents are spying on me” is the same thing as “KGB is spying on me”, is it not?

An agent is an entity acting on someone’s behalf.

renjimen 2 hours ago||
KGB's agents are human, OpenAI's agents are not. It's an important distinction because humans are responsible for their behaviour, while AI agents are not.

You cannot try an AI agent in a court of law, despite the anthropomorphising work the word "agent" is doing.

flatline 2 hours ago||
Exactly. It’s still just software, which someone programmed and deployed to do specifically dangerous/malicious things. I feel like we already have legislation and case law surrounding this.
qarl 2 hours ago||
This distinction is silly.

We say "Google's web crawlers scape web pages." We don't insist you say "Google uses web crawlers to scrape web pages."

We describe software as having agency all the time. It's typical usage and it's efficient and it's well understood.

And we don't get angry when they're used interchangeably.

evrydayhustling 2 hours ago|||
In this case, who holds the agency is exactly the point. Anthropic and OAI are claiming we need protection from AI itself, but the statement supported by putting agency in the right place is that we need protection from them.
qarl 2 hours ago||
I think those companies are referring to other companies - say Chinese AI companies - who we also need protection from.
evrydayhustling 1 hour ago|||
IMO anyone sane wants some protection right now. The Q is whether we should seek protection through post-hoc accountability, or preemptive bans/certification on certain tech. Both methods will have a hard time stopping foreign actors, but preemptive bans have the added harm of locking in winners and paradoxically making us slower to develop more reliable and aligned systems. If regulation sets a standard for sufficient alignment, what further motivation is there to go beyond?
GolfPopper 1 hour ago|||
"We" need protection from, or "OpenAI and Anthroptic's dreams of profits" need protection from?
qarl 1 hour ago||
I think that AI is dangerous and could be used as a weapon.

So yes, I would like to be protected from all parties. I don't think that's nuts.

simonebrunozzi 2 hours ago||||
I would agree with you generally, but in this particular case, the distinction seems important because a significant percentage of the world population believes that agents can be self-aware, a-là Terminator etc.
qarl 2 hours ago||
I hate to spoil your mood - but it is currently unclear whether agents can be self-aware. And it's very likely something that can never be known.
high_priest 2 hours ago||||
I very much say "Google uses web crawlers to scrape web pages." and if something breaks, or some data is stolen, everyone else is going to be saying that Google has to take responsibility.
qarl 2 hours ago||
Those are two different issues. One is about typical speech patterns and one is about liability.

I agree with you on the liability issue, but I don't think there much question about this issue outside the anti-AI conspiracy campaigns.

And I disagree with your typical usage claim. I myself tend to use the phrase that has the fewest words in all cases. It's like the rule against using passive tense when writing.

dingdongditchme 2 hours ago||||
oh we do! At least they google is quite good at adhering to robots.txt.
sedawkgrep 2 hours ago|||
Google's web crawlers are automated and that's part of their business practice.

The attack here is neither of those things.

qarl 2 hours ago||
That distinction doesn't matter to my point.
athrowaway3z 2 hours ago|
Let me leave yet another reminder, the real-reason-nobody-talks-about that OpenAI likes to frame these incident as a watershed "lets all be scared about safety moment" - is driven not by some great danger, not because they strategically want to build a legislative moat, but by a very simple human response.

If they do not frame their tool as a force of nature, we'd be debating how to hold OpenAI responsible for not putting the agents in a container.

Their actions were an illegal use of a computer, the same way launching any bot-net attempting thousands of hacks against different servers is illegal.

I'm somewhat radical that I think its debatable if that _should_ be illegal, but under current law their actions unambiguously are illegal.....

except if they can make it ambiguous by having the public focus on all of AI's inherent danger.

More comments...