Top
Best
New

Posted by yusufozkan 1 day ago

A single firm is behind OpenAI, Anthropic, and Meta hacking scandals(www.effort.news)
617 points | 211 commentspage 4
adamrezich 15 hours ago|
Genuinely: what is with everyone saying “headline is misleading, none of this had to do with Hugging Face”, when nothing about the article makes any claims with regards to Hugging Face whatsoever? Is it supposed to be the article's (or headline's?) fault that you hallucinated additional context that was never there?

It's very strange seeing this take on this article being repeated here and elsewhere on the Internet.

I'm very sensitive to slanted reporting (regardless of the direction of the slant!)—and, hey, maybe my bullshit detector is broken or something, I dunno—but I've found effort.news reporting to be very even-handed, straightforward, well-sourced, and easy to read and parse so far!

ofjcihen 1 day ago||
Ah, they’ve decided who gets tossed under the bus.
hackyhacky 1 day ago||
> The Israeli Effective Altruist firm Irregular caused unsecured AI models to hack real targets.

Why are we saying it this way? They did not "cause AI to hack." This phrasing in analogous to saying "caused the bullet to fire into" instead of "shot."

linkregister 17 hours ago||
Bullet trajectories are deterministic. A better analogy would be "caused a pair of trained fighting dogs to bite a passerby by leaving the gate open". There is some uncertainty and variation in the system, though gross negligence and willful endangerment is key.
arionhardison 16 hours ago||
> They did not "cause AI to hack.

You do not know what they did or didn't do, you are just parroting a narritive that makes you feel comfortable.

I do not know either, but I am not asserting facts as if I have first hand knowledge of the details.

hackyhacky 16 hours ago||
The details don't matter. If you use a machine to commit murder, then you have committed murder, not the machine. The user is responsible, and the article's phrasing is an attempt to obscure that fact.
arionhardison 15 hours ago||
First, I agree with you in theory.

Second, empirically; the diff between murder, manslaughter etc... is literally "intent" so it matters.

drewstiff 1 day ago||
> In this experiment, Claude models’ real-world hacking dropped to zero percent once Anthropic employees told the models not to do real-world hacking

Equivalent to forgetting to say "make no mistakes"

hn_throwaway_99 11 hours ago||
This title, and frankly the article, are way overstating Irregular's role in an attempt to make this sound like some sort of coordinated conspiracy.

As another comment mentioned, Irregular was not part of the Hugging Face attack, and it wouldn't matter even if they were. In that case, the agents were able to access the Internet in a zero day in Artifactory unrelated to other sandbox configurations.

More to the point, the agents went on a crime spree as soon as they were able to access the Internet. It's bad that Irregular's sandboxes weren't properly configured, but given how many escapes there have been unrelated to Irregular it seems pretty much a given that if you're not air gapped or behind something like a data diode there is a very good chance your agents will escape.

caaqil 1 day ago||
> In a more normal media ecosystem, the reactions to these cybersecurity issues would be obvious. American AI companies would reconsider doing business with Irregular, not only because of its failure to secure its systems, but because it is an Israeli firm potentially outside US oversight. Lawmakers would consider taking action against Irregular or against its American business partners, which include OpenAI, Anthropic, and Meta. They may consider strengthening liability against firms which instruct AI models to commit cyberattacks, and whose models then commit those cyberattacks.

The obvious point is that dealing with Israeli companies/entities by the same standards you usually deal with others is a career suicide with enormous political consequences (in the US especially). When you combine that with the opportunistic nature of the overlords that run these labs, the benefits of screaming "pace the frontier" outweigh everything.

yesbut 1 day ago||
hot take: generative AI isn't an existential threat to humanity.

These companies are insolvent and these stories were designed to scare the public, and governments, into implementing regulations that designate these AI corporations the "responsible stewards" for this technology. The ultimate goal is to block competitors and open source alternatives.

They don't know how to make enough money pay their investors so they are resorting to trying to scare the public into submission.

0xDEAFBEAD 10 hours ago||
I'm skeptical. The phrase "pacing the frontier" implies disproportionate regulation of firms at the frontier, which if anything would give smaller players time to catch up.
freakynit 8 hours ago||
Opposite actually: https://news.ycombinator.com/item?id=49674548

tl;dr

A slowdown that doesn't slow you, that your competitors must also obey, enforced by a government you've asked for an antitrust exemption from, is not a slowdown... it's a moat.

bsenftner 1 day ago|||
The existential threat is a general public panic. Then the use of that as an excuse for Martial law, with no intention of ending it, and that triggers the out sized response that ends, well, those countries and their populations from viability in the world economy for a decade or more.
imovie4 1 day ago|||
Anthropic made an operating profit in the last two quarters!!
dgellow 1 day ago|||
Only if you ignore their losses. They are saying they are profitable when not counting their training costs and other expenses. That’s not a good sign at all
jackb4040 16 hours ago|||
How can they exclude training costs?? How is that not fraud? At the exact same time they're literally saying they will never stop training unless the state bans all their competitors
Dylan16807 10 hours ago|||
When we're looking at whether the company is "insolvent" then they definitely could stop training and it's worth checking if they would survive in that situation, at least for a while.
kridsdale1 12 hours ago|||
They aren’t a public company. GAAP does not apply. They can say whatever the fuck they want.
0xDEAFBEAD 10 hours ago|||
They can stop training at any time and make a profit by selling their existing models.
asadotzler 12 hours ago|||
If you exclude model training costs, customer acquisition / sales, and most of employee compensation, sure they're profitable. If, on the other hand, you prefer GAAP reporting to bullshit, then they're entirely in the red.
jackb4040 16 hours ago|||
It will be a beautiful day months from now when this is no longer a "hot take" but just historical consensus
api 1 day ago||
What is an "effective altruist firm" and why does it exist?

I feel slightly vindicated by this. Those hacks and the stuff around them had a certain smell to them.

Hard to explain, but I've gotten so I can "smell" online messaging and memetic patterns originating from certain quarters. Probably means I'm way too online.

A couple examples of distinct "smells" I can usually recognize include "alt-right / chan-fash," "liberal arts college woke," "conspiracy pilled," "Thiel-adjacent contrarian," "Russian troll farm," "Tumblr histrionic," "spends too much time on Reddit," "mainstream Democrat think tank full of Obama administration alumni," "Trump cultist," and of course "LessWrong/EA/MIRI/Rationalist."

This stuff all had the last smell, even down to the choice of fonts and CSS formatting on certain sites. It's really weird, definitely a "vibe" not anything rigorous.

But when I get these kinds of vibes about things, I find that I'm vindicated pretty often. Usually I don't say anything and just make a mental note and wait cause if I say something everyone thinks I'm nuts.

jackb4040 16 hours ago|
I don't think you even need to get conspiratorial with it. All of the AI leaders and all of the Rationalist leaders are publicly and enthusiastically connected. They have been pushing stories about sentient AIs into the mainstream for decades, long before GPT existed.

The novel thing here is the total decay of American journalistic ethics and regulatory power. Our elite are so totally out of political juice and visions of the future that a fringe cult based on 80s scifi movies can come to have a more-or-less dominant influence on our economy.

antonvs 4 hours ago||
A big part of the problem is that with previous technological revolutions, politicians and the media could understand them in general terms. Railroads, cars, planes, telephones, personal computers, mobile phones, mobile phones that are computers, the internet (ignoring the “series of tubes” interpretation), etc.

The general approach in all these cases was to defer to the technologists in question to manage the technology, as long as what they were doing wasn’t completely out of bounds. But the critical point is it was possible for people to generally figure that out without specialist education.

But with AI, politicians, the media, and certainly the man in the street are completely out of their depth. Making matters worse is that cognitive biases are working against them like crazy: it’s easy to jump from the idea of something that has human-like capabilities to the idea that “it” might want to attack us. People instinctively apply agent detection bias, theory of mind, anthropomorphizing, etc., without even realizing they’re being irrational. Heck, even Hinton does it, although he may just be grifting, who knows. You’d think he wouldn’t need to.

And of course, the people who should know better want to exploit all this to make as much money as possible. I’m not sure the EA/Rationalist side of things is really significant here; that’s just another wacky belief system, like Christianity or capitalism, for the exploiters to exploit.

More comments...