Posted by snikolaev 7 hours ago
Now I think the correct response is both trying in court to stretch CFAA and state statutes to cover, which will be highly fact specific, and update the law.
But in either case won’t be a slam dunk.
PSA to folks in the thread: If you’re American call or write to your state and Federal reps about this, and if not investigate whether there are gaps in your country’s laws.
[1]: https://en.wikipedia.org/wiki/Computer_Fraud_and_Abuse_Act
EDIT: See for example...
The Computer Fraud and Abuse Act (CFAA), the primary federal statute governing unauthorized computer access, was written decades ago with human intruders in mind. Its key provisions require intentional or knowing unauthorized access (a mental state that maps neatly onto a person who decides to break into a system), but what happens when the hacker is an AI model that selected its own target?
On the current facts, CFAA liability for OpenAI is unlikely.
Almost all common felonies require specific intent. Misdemeanors often do not.
There is plenty of civil liability available.
If you wanted them to be charged with a felony you would need changes. I would strongly suggest you do not want a strict liability felony.
The cfaa required intent is as follows :
* § 1030(a)(5)(A): knowingly transmits code/commands and intentionally causes damage without authorization.
* § 1030(a)(5)(B): intentionally accesses without authorization and recklessly causes damage.
* § 1030(a)(5)(C): intentionally accesses without authorization and causes damage and loss;
Simply changing the first intentionally to intentionally or recklessly would cover OpenAI (now that they know it can occur) without causing lots of other issues
Only in terms of CFAA, not in terms of damages. Culpability does not require intent.
You may not have intended to attack $CORP, but you can still made to pay the cleanup costs of that attack.
So, yeah, you won't be convicted, but current laws still allow for you to be billed.
With that said, there is also criminal negligence. Now that OpenAI is made aware of the risks, it's also expected to take additional precautions in the future, otherwise there could be criminal liability as well.
I think security will suddenly become much more important.
It's not my first prize, but I won't mind it. And millions like me won't mind it. Easy way to make money - setup a site with all the default server software installed and patched at a reasonable frequency. Then just wait for bots to attack it, and claim a few hundred (or single-digit thousand) dollars from OpenAI or Anthropic, etc.
Sure, it's pocket change for them, but just the admin of dealing with millions of cases will, even if they win half the time, will bankrupt them. Thus, they have incentive to make sure that their bots are not performing attacks.
First prize is, of course, holding them liable with punitive fines, not theatrical fines.
Negligence would be interesting given the grand claims of capability of AI models from the AI companies and their executives. If they believe the claims, why not much stronger precautions?
You cant just copy existing work and feed into machine and just pretending its not violating copyright
So as long as there's no motive behind it then it's just OK?
Funnily enough the US already has one similar real argument around guns - should gun manufacturers be liable for damages caused by their product?
The question is already settled - gun users are responsible for damages arising from their usage of the guns.
Why would AI users not be responsible for damages arising from their usage of the AI?
Because, as usual with that kind of question, it's not that simple.
Let's say an user asks ChatGPT to get some info about something and for some reason it starts using exploits in the background to get them from a server. Should the user be responsible or OpenAI?
Okay, lets go with that as scenario #1.
For scenario #2 lets use "developer asks an agent to a self-hosted LLM to get the docs for a ERP system, and it hacks the vendor to get unreleased and undocumented docs".
We'll assume, for the sake of this argument, that in neither case did the user intend for any malicious action to be performed.
> Should the user be responsible or OpenAI?
In scenario #1, the agent+LLM is under the control of OpenAI, not the user, so OpenAI is liable.
In scenario #2, the agent+LLM is under the control of the user, so the user is liable.
There is no scenario anyone can come up with that is not addressed sufficiently by existing laws[1].
It's very clear, and it's only getting muddied because there's a group of powerful people who want exemptions from the current law.
IOW, the only reason to draft new laws for AIs is to exempt their usage from the current laws.
========================
[1] Possible 3rd option (local agent + OpenAI LLM). In that case an investigation would determine where the culpability lies. Just like how it is currently done in law.
When a pressure-cooker explodes and kills someone there are only two possible liable parties: either the user or the manufacturer. An investigation determines who's liable. I see no reason to automatically exempt everyone from liability just because an agent did something.
I have lost track of the metaphor, but man pressure cooker lawsuits are more common than I thought.
I don’t think OpenAI or any large company will see more than some fines and new legislation but only after a disaster.
AI assistant hacks gym website in first known Australian autonomous cyber attack: https://www.abc.net.au/news/2026-08-10/ai-assistant-hacks-gy...
General opinion at the time was it was in fact ambiguous who was legally liable.
You are going to jail.
Like hypothetically speaking if autonomous cars get taken over by an OpenAI rogue AI and it starts hunting down Anthropic employees who is to blame?
Even without intent, there is still liability.
> Now I think the correct response is […] and update the law.
Essentially we need some enforceable equivalent of gross misconduct or, to be a little more hysterical, manslaughter & culpable manslaughter. It will need to be globally, or at least very widely, enforceable to be truly effective thought, good luck getting that arranged before the need is so far evolved that we need to respond with something else entirely!
Actually... if you combine https://news.ycombinator.com/item?id=49827099
>Since the publicized AI agent hacks typically aren't malicious, maybe it's time to start plastering all public facing web infrastructure with polite requests to stop hacking. Nothing to stop three letter agencies though.
with automated delivery of cease and desist letters, you can retroactively establish intent on the operator of the agent since the autonomous agent system must acknowledge the cease and desist letter in their autonomous pipeline or the operator must argue for their own willful ignorance or negligence with regards to cease and desist letters. The fact that they used an agent on their behalf to ignore the letter is irrelevant.
My money is on special teams co-ordinating these agents and exposing their traces in order to create a pre-ipo buzz. Sounds ridiculous and reckless? Well that’s the AI industry for you in two words.
1. What about negligence?
2. Every follow up to every story after the news cycle moved on shows both intent and negligence. To the point of "we opened internet access and told it to hack"
In this case, it's not even about the people driving self-driving cars. It's like someone launching a car into traffic just to see what would happen. Even Tesla puts a human in the car when they do their self-driving trials, it's almost impressive that AI companies have somehow managed to out-neglige Tesla.
If you're arguing that they should be put on trial for negligence, that's fine. It does seem we're moving towards open source being outlawed, or at least the end of "no liability" clauses in open source & freeware. Just make sure that is the result you're advocating for.
[For the future record: at the time I am posting the link below on 24 September, it has 1 point, no comments, and the poster has a karma of 1. This is not an active HN user, or a ShowHN project that had traction, beyond seemingly OpenAI's swarm.]
If the developer behind ShotAPI had started letting the ShotAPI code take shots at the Austrlian government then yes, ShotAPI (or rather, the people behind it) would be responsible.
Blaming ShotAPI would be like blaming OpenAI for what its users are doing. That's not what's happening here. And if ShotAPI did knowingly let its users somehow hack the Australian government, then maybe they should be investigated.
Owning a gun, writing articles about how powerful and dangerous your gun is, then making deals based on ability of your gun to kill people, and then getting completely astonished that "my gun killed some people, completely bonkers! (invest now)". It's not possible for the selling point of your product to be unintended.
If you walk out onto a busy street, pull out a gun, close your eyes and start randomly shooting around you until you hit someone, you don't get to go "whoops, didn't mean to" afterwards, it's still murder.
So, I’ll ask a controversial question: is any hacking so problematic to make a big deal of it?
But I do not think this is misguided. They never publish the harnesses and the models so they are not inspected.
In this world where oligarchs are immune from everything, it's a lot less clear.
Blaming OpenAI (or Claude or X-whatever) would mean blaming powerful rich people, so that will never happen. Some poor person with no influence will go to jail instead.
You want AI labs to pace? Simply hold them liable for their products.
But they're not even talking about that, either.
I agree they should though.
This isn't even "an openai customer tried to hack someone", which can be defended. This is the AI companies themselves fucking around.
If new weapons still operating inside any of these companies spew a million bullets on my house, they are still liable. Humans are setting these system up and they still have to behave responsibly.
A better analogy may have been the troubles Meta has faced around child protections on their platforms. Technically the abuse and problems have stemmed from individuals too, but they've in many respects enabled the situation by failing to moderate or flag warning signs. OpenAI is failing to moderate the models in similar ways.
It's irresponsible for OpenAI to give unaligned agents a prompt to 'go hack' and internet access. They know better, so I am thinking they might have other intentions to let those swarms have any sort of internet access.
> Much of the urlquery.net activity appears to come from agents retrieving data to answer web search tasks. For three of these tasks, after failing to retrieve data through normal means, they attempted a variety of cyber exploits against the relevant data service... This data reveals that malicious cyber activity is not limited to agents tasked with cybersecurity-related tasks and can arise instrumentally to solve mundane tasks like information retrieval.
And you are already assuming that OpenAI is intentionally using unaligned agents in these evals or training runs or whatever it is that produces these breakouts. But what if the problem is that none of the alignment techniques that are applied to models today actually work? What if all the agents involved in these incidents have in fact had the full stack of alignment applied - isn't that a good reason to regulate any high-compute usage of models, as the Klein crowd is proposing?
If I run a biology lab and engineer a terrible virus, it gets out, and a global pandemic ensues, I don't get to shrug and say "well we told it not to infect people". It's my fault for failing to mitigate the risks of my work.
A big part of this developing story is that it happened during training of a new model that ended up misaligned. And training happened without the usual safeguards applied like chain-of-thought monitoring. So OpenAI has already admitted that the full stack of aligment had certainly not been applied in this case.
Also, my understanding is that the models involved in the HuggingFace hack did go through the full alignment training; they just didn't have the classifier that normally prevents hacking attempts.
I was referring to the HuggingFace incident.
> none of the alignment techniques that are applied to models today actually work
none of techniques to autonomously drive a car was/is not working for a long time. no company came out and said 'this is impossible to do, let's change the regulations'.
It doesn’t matter, and the legal entity in here (the AI company) is liable. If a robotic company built an autonomous system or a robot to do certain things in an autonomous ways (not predefined) and these systems are starting to kill people, that company is liable regardless, you don’t blame the robot or the autonomous system, but whoever made it
> If you find two ants in your kitchen, the best estimate of the total number of ants in your kitchen is not two
What LLM's hacking isn't is AI acting maliciously in any kind of sentient way. Its just the code behaving how its always behaved but now it has better tools to navigate the web. This has literally been happening this whole time.
By the way - LLMs aren't code. They are not designed by humans; they are grown, in a process not dissimilar to evolution except much faster.
Why is OpenAI getting away with crimes?
If you drive drunk and you have an accident that alcohol may be a factor but you are at fault.
There are no "rogue AIs" just irresponsible corporations.
> There are no "rogue AIs" just irresponsible corporations.
If you have a prison and prisoners escaped, these are rogue prisoners irrespective of whether you were irresponsible or not.
I suppose you could call these highly goal-oriented autonomous agents "tools", but this does sound like playing language games.
> A rogue is a person or entity that flouts accepted norms of behavior or strikes out on an independent and possibly destructive path.
Read the [HuggingFace incident report](https://metr.org/blog/2026-08-26-openai-hugging-face-inciden...) to understand how these attacks develop.
The METR investigation, which you evidently refused to read, is a third party investigation of the HuggingFace accident. One of the investigators has even participated to many interviews. It's mind-blowing, and it's extremely evident how it developed.
But some people think the moon landing is a conspiracy, so I'm not surprised.
2.) And HuggingFace accident is exactly accident where agents trained, prompted to hack hacked and tested on their hacking abilities hacked, due to sandboxing failure.
3.) If they in fact have roque agents, they themselves should be first to stop. Not trying to make legislation for others, they themselves are bad supervisors. All it requires is to stop electricity for data centers.
However, we know (independently to OpenAI/Anthropic) from the incident at AISI that the models can hack things without human intention if they happen to also have internet access (which in reality all agents in deployment have).
https://www.aisi.gov.uk/blog/incident-report-unsanctioned-ag...
Yes, the monitoring guardrails were off in that incident - but if that is the only protection, we need to require all models are behind regulated APIs, not open weights, and not served from providers who aren't monitored.
If we assume that rouge agents actually exists, then OpenAI needs to shutdown EVERYTHING, right now. My personal take is that OpenAI, and maybe Anthropic, desperately wants someone (e.g. the government) to tell them that they need to stop/pause/slow down. They are bleeding cash (especially OpenAI) and needs a knight in shinning armor to swoop in a pull the breaks, so that they have an excuse to investors when they need to explain why they need $50B more next year.
When it was unable to, it used cross site scripting as a way to check the capabilities accessible through the browser making the requests. In this case cross site scripting wouldn’t be a hack against the Australian website, it would be a hack against the urlquery site, if one could even call it that.
Finally, downloading public files from the public pre-production server also seems like a non-issue.
The sql injection attempts against the other sites are less ambiguous. Attempting to access non-public user passwords rather than reasonably tweaking the parameters for a site designed to serve public data are categorically different things.