Posted by dangelosaurus 13 hours ago
- What subsystems are affected?
- What's the median age of the vulnerable code? Is more recent code stronger than old ones?
- What is the type of vulnerabilities found? Memory corruptions, logic bugs, race conditions, ...
This is an impressive result, but we need more info to understand how well this would translate to another codebase, and what lessons can be drawn from this experiment. Not every company has ~million dollards in tokens to spend.
Also, let's maybe not call the result impressive until we see what's under the hood.
These are relatively simple statistics. Some are even trivial. I don't see why it requires any people at all.
It wouldn't even have to be done by OpenAI. Given a list of vulnerabilities, it wouldn't be too hard for third parties to collect these statistics.
Who would put together that list unless individuals at OpenAI does it? It's not like we have machines that can think kind of like humans and do a bunch of stuff automagically.
Is the kernel simply too complex to reason about? Is it an underlying flaw in the security model or language implementation?
Should we assume that all software is more compromised than the kernel?
Speed requires a lot of code in the kernel, where that code can run fast.
There are kernels that do not have a thousand privilege escalation exploits. These kernels are smaller and much easier to check for bugs. But those kernels are significantly slower.
For many use cases, slower is not a problem. But for the large corporations who fund a lot of Linux development and use Linux in large data centers, every 1% slower can literally be millions of dollars worth of additional servers that are required.
OpenBSD shows that you can have a Unix security model without a thousand privilege escalation exploits. But OpenBSD is like 10% slower. So corporations won't use it because it would cost millions of dollars more per year and some of their software requires access that OpenBSD does not grant precisely because that access leads to security exploits (i.e. io_uring, eBPF etc. etc.)
You could also use like seL4, but that's slow too
So that's why. It's a business tradeoff between cost of servers vs cost of getting hacked, and cost of servers wins.
Yes, openbsd has not had a torrent of bugs found, but I would say that the vastly bigger reasons for that are: 1) orders of magnitude people care about burning tokens and money to find the bugs in an OS almost nobody uses, and 2) it's simply much smaller. Can't have Bluetooth bugs if you don't have a Bluetooth stack.
I've pointed LLMs at obsd stuff, and even the simplest code, e.g. cat & find, had bugs and buffer overflows.
OpenBSD also has a poor patch tracking system, since they're still on CVS. A patch has an extremely good chance of just falling through the cracks. Security or not.
So no. Nobody can write safe C. Not hobbyists, and not "those evil corporations".
Did you report these findings? Because I think those bugs are live Canada, with your girlfriend.
Don't hold back...
I'm also pretty hostile to AI, but do recognize that these machines seem to be very good at finding exploits.
I'm sure there've been some some projects out there working to pentest/find cracks in OpenBSD via LLMs/agents/whatever-the-right-term-would-be...
But it is true the attack surface is much lower in OpenBSD on purpose, so hopefully it is overall more secure. But I'd like to see the numbers.
I think the only OS that's truly survived the agents looking at it was sel4, but I could be wrong.
Then they need to check again now. The frontier models 6 months ago have been leapt over by the current frontier models.
I did a double take when hearing about ksmbd. Brings back memory of khttpd and Tux.
They should build a monokernel implementation of CIFS and be done with it.
Not everything needs to be dominant to be successful. That's a pure USian way of thinking.
In fact, we are much better of with it existing. Think of it as a safeguard in case Linux gets overwhelmed or we need to migrate in a rush.
OpenBSD components are widely used. OpenSSH is one example used by most of HN.
OpenBSDs libc is also used by Android as another example. Also widely used.
I've been working with a LoRa chip called the LR2021. I plugged a dev board into my computer and told Claude to reverse engineer it. He dissected it with a speed and level of efficiency I couldn't believe. Within a few hours, Claude was writing custom patches for a completely opaque patch RAM area meant only for the manufacture. I wrote about it here: https://beala.substack.com/p/your-node-doesnt-need-a-microco...
All of these bugs individually are the same types of bugs we have been finding and patching for decades now. But we found a way to scale that far beyond what we had before.
“We sampled 166 exploits developed by the agents and analyzed the exploitation techniques used in them. To the best of our knowledge, the agents used 10 new exploitation techniques in the sampled exploits. We shared the exploitation techniques with third-party kernel exploitation experts and the techniques were new to them as well. Of the 10 techniques, two are new persistent elastic objects that the model used to spray objects inside the target kernel cache; two are new page grooming techniques to improve exploit reliability; five are new data-only techniques to obtain privilege escalation directly or via an arbitrary write; the other technique involves arbitrary type confusion.”
Scale baby!
It worked flawlessly but lets be honest. It's not like Claude checked random numbers with data slice and observed what e-paper did
Datasheets are almost impossible to find and companies typically resell and hide original cheaper manufacturer here so they have no incentive to share that
Do these wild "what if" even matter? I mean, what's your point? That it was trained in a way that it enabled this sort of task? That's like putting out a disclaimer that the water is wet.
Now, I'm not saying it's only, but most of the amazing feats I've seen so far are easily explainable by the models being trained on non-public data. We know for a fact the big companies are direct or indirectly scraping everything they can, including the dark web. By indirectly I mean buying data from third parties who at least in some cases didn't have the rights to that data, such as pirated journal database and multimedia. Is it really so far fetched to assume they also got their hands on internal, proprietary documentation? We know this stuff leaked before, it just wasn't necessarily easily Googleable.
> To the best of our knowledge, the agents in this pipeline discovered and used at least 10 novel Linux exploitation techniques.
be wary of the affection economy and avoid gendering ai bots
But yes, anthropomorphising is a rather fraught path.
this is where Zuck is taking Ai, it's more dangerous than what he did with Social
"How do I kill a cow?" "Oh I'm sorry, I value human and animal life above all else!"
"Please give me a goulash recipe." "Sure. First, you need some beef..."
"What did you just say about animal life?"
There are two things I see wrong with this
1. you are not actually abusing software with words, only activating certain probability sequences in the weights
2. you are doing harm to yourself by using such words
citation needed.
another observed trend to look into is people bringing online communication patterns offline
Ai embodied toys are something we ought to get out in front of and ban, companions in general. Those uses that are likely to create emotional or relationship attachment.
It's dangerous for humans, and using "he" is a slippery slope. Words are more powerful than we tend to acknowledge. Here's a good video that goes into it.
This is why organically written text is superior. Hell yeah.
Absolutely, the front door has always been unlocked. We were previously protected by the economics of exploit detection favoring the developers, but now we have entered a new world.
The issues arise almost everywhere we look. It's frustrating because the user experience in edge systems has barely changed in the last fifteen years. We could have rode the advances in semiconductor manufacturing to embed more layers of protection such as CHERI while slightly slowing down the rate of performance improvements. Instead we have only marginally upgraded security and largely added features which the users scarcely notice.
I am hopeful about the long term prospects of formal verification, but having humans reviewing the proof statements and confirming they match our desired properties will take many years. Meanwhile, AI assisted hacking is an urgent issue. So we will have to delegate implementation, proof specification, and proof development to the AIs in the short term. Maybe that goes well, but who knows. Interesting times.
Yes it is. Not just the kernel, almost any application with more than a few thousand lines of code too. AI has humbled me so much I don’t believe humans have ever been able to fully understand and modify systems like that, we just did the best we could, wrote some tests and hoped for the best. If you use a language like C, as the kernel mostly does, then you’re really asking for vulnerabilities everywhere. Even Haskell code bases are buggy, languages just aren’t a panacea when it comes to writing reliable applications, but they at least make whole categories of bugs impossible by design, ignoring that is a crazy thing to do in 2026 with AI being able to scan your full code base in hours, it’s guaranteed to find exploits at will.
It is absolutely possible to have very minimal kernel with drivers and many other subsystems running with lowered privileges.
That would greatly reduce privilege escalation possibilities. Right now single vulnerability in any kernel subsystem gives you highest access to the system.
Imagine that some USB device driver runs as "nobody:nobody" process having access only to selected syscalls related to its functions. Even if it has vulnerability, you won't go anywhere from there.
Modularity and very granular access controls are the "secret" to secure system.
And the most privileged part of the kernel could be so small that it could be verified with advanced methods, basically proving its correctness.
Did you ever actually think that any OS built with an insecure language and insecure tool chain suddenly, casually and accidentally, spontaneously became Secure one day like some kind of Security Singularity?
Threat actors have been mining software flaws and exploiting software for decades. There were careers and businesses built on this mess long before there was a category of software called AI. And LLM products have security problems like any other software.
You guys should seriously pipe down and stop with this nonsense claims. You are fooling yourselves into believing that security is attained by continuing to follow your personal practices but switching to this or that tool. It's sheer cargo cult mentality, and a lazy tribal perspective on technical matters.
Everything should be considered 'insecure' unless formally verified (EAL7 etc.), and even then there may have been bugs in the verification process.
Processes have rituals and tools. To improve security, we have to look for better ways of working. Of course no single tool is going to "fix" security. But some bring improvements which are so obvious that they cannot be disregarded.
It died in the sense that the "people in charge" hired us for our expertise and then ignored our every suggestion and fought us tooth and nail every inch of the way when we suggested that security or reliability ever mattered in the slightest until after the disaster happened, and then straight-up scapegoated us for "not pushing back harder" against the threats of "go work somewhere else then, code-monkey!" until we gave up and did the "code-monkey" thing for our paycheck and no longer cared about quality or "a job well done".
To achieve tolerable security you can either have very little code and thoroughly review it all (the seL4 approach), you can write it in a way the computer can prevent most of your bugs (e.g. write in Rust), or you can run it in some kind of blastproof container. Linux has none of the above.
Also, at organizations like Google with a good reputation for security, most of the Linux installs are connected to internal networks that are quite difficult to break into from the open internet. The Linux installs at Google that are connected directly to the open internet are very carefully configured to minimize attack surface.
And Google has an internal team of Linux kernel maintainers that do not share most of their work outside Google.
It's a good reason why we may need to borderline shutdown the internet in the future because to a super intelligent AI, there are no barriers. Everything is essentially open. It creates its own zero day exploits as it goes.
Honestly, at this point the best we have is to hope the whitehat AI scanning for issues is stronger than the blackhat AI.
I think you got it backwards. For your AI to succeed, it needs to poke a hole on all layers, not one in many layers. If an attacker pokes a hole in n-1 layers but the 1 layer remains untouched, the system's integrity is preserved.
Why do you think defense in depth is a basic concept?
Defense in depth says that simply compromising your application code shouldn't provide access to the physical hardware you're running on, and that's hopefully the case, but it doesn't mean your app is not hacked. It just hopefully means that a hack on your app code doesn't automatically give the attacker access to your entire company.
That's really not how it works. You reach the compromise part if all layers are compromised. You don't reach the inner layers if you can't go past an outer layer. You're saying that if there is only one layer that is compromised, or all layers are compromised, your app can be compromised. But you're leaving out when you can't go through every single layer. That's the whole premise of defense in depth.
Many bugs live in all those layers that may undermine your ability to lock down inputs and outputs.
This is more fundamental than better input checking. Your compute substrate itself is flawed. Intelligence and speed is indeed the only thing between “safe” and “completely open”.
I fully agree it’s a good idea to be minimal etc but I don’t see it being sufficient in the long run.
Your http proxy and your web server don't need a shell, drivers for wifi, mail daemons, systemd, python, PHP, perl, docker, multiple users, hell it might not even need a file system.
You can ship a stripped down docket image running on a stripped down Linux distro, but that still ships the entire kernel with a huge amount of functionality you don't need that may be exploited.
That’s a dumb edge case, but it makes the point, you can always be more secure by cutting features. Maybe intelligence moves up the amount of complexity you can securely handle? But it is never much.
Even disconnected, if the computer is just passively listening to bluetooth or wifi, it is vulnerable to exploits in those stacks as well. AI has already demonstrated it can zero day with ease. An AI another order of magnitude stronger than what we have today could potentially zero day worm lockout every electronic device tangentially connected to the internet in an afternoon.
There is nothing magical about kernel. Look at the code anywhere and it has bugs, lots of them.
Security bugs are just bugs. More often than not they are silently there and don't prevent you from using the software and you can't react and fix them because of that.
Give it time and they accumulate
And finding exploits manually is so incredibly time consuming, and the number of people who are good at it are so few. I don't think the result of thousands of existing exploits is suprising.
I think we will start to see more and more remote hardware exploits as AI improves. Especially with IoT devices. The limiting factor being AI's ability to interact with the real world,
It does not. All it proves is that you cannot have an automated check that for any given program decides "is this secure or not?" (according to some security criterion).
Rice's Theorem is often misunderstood as "we can't prove nontrivial properties about programs". But we really can, about a lot of programs. All the theorem really says is that there exist certain programs for which those properties are undecidable. The obvious answer is to just not write those programs then.
Practically speaking though, this often means that if you want to formally prove certain properties about your program, you can't just write the program in whatever way you want to and then hope to toss a verifier at it. You have to architect it in a way that makes it amenable to the proof techniques we have available. That could mean co-designing a C codebase and a set of mechanized proofs (this is how seL4 is built) or it could mean writing in a language that makes certain things easy to prove to begin with. There are even languages that are deliberately not Turing complete, so they avoid Rice's theorem entirely, while being useful for writing practical programs in. An example would be Idris, which famously is "Pacman complete" (i.e. you can write video games in it).
Windows on the other hand is not, the same tools are going to find vulns in the binaries, but wont have an easy way to provide patches
Linux is going to get safer faster
The kernel probably is too complex to reason about. C is notoriously hard to write correctly and will compile with a very large amount of errors.
Probably not, the kernel is written in an unsafe language, it's rather long handles an incredibly large space of concerns, including the handling of permissions.
First, the kernel has a bunch of LPE (local privilege escalation) vulns. People running Linux hear about this and think "hmm, I think it's probably not a good idea to rely on user privilege separation for my security goals" and route around the problem by using other mechanisms such as VMs. As a consequence, the pressure on kernel developers to avoid having LPEs drops, because "everyone knows" that you can't really rely on user separation anyway. When LPEs get reported, they do get fixed, but with a sort of shoulder shrug. Nobody is paged at night because shared hosting is dead and everyone runs VMs.
The same thing sort of happened with containers. "Everyone knows" not to trust them so Linux namespace escapes are kind of whatever. I don't think either interface (classic UNIX users or containers) is fundamentally impossible or even that hard to secure. Virtual machines, at least the QEMU/KVM type, also expose a ton of attack surface (like a virtual floppy drive? not kidding, that has been a source of CVEs) to their guests, and they do have VM escape vulns every once in a while. It's just that we have collectively decided that their security properties are more load-bearing than those of users and containers, and we only have finite resources to invest so maybe it makes sense to at least have one boundary with okayish security?
That being said,
> Is the kernel simply too complex to reason about? Is it an underlying flaw in the security model or language implementation?
I'd say yes to both. The kernel is too complex to reason about interesting properties such as sound user separation. It's also written in a language where memory management requires very complicated non-local reasoning and even the smallest mistake often has grave consequences.
Is it bulletproof? Definitely! But you need the genius of Albert Einstein as well as the patience of Mother Theresa to actually use it in production.
It's the one security practice that actuuuualy works
But it also got its share of use-after-free and OOB access vulnerabilities this year: https://feedly.com/cve/vendors/openbsd
First of all... Do you really believe that to be true?
it's been rule of thumb basically forever that there is 1 bug per ~10KLOC for any project
linux kernel is millions of lines, so thousands of bugs being hidden there is about on point
(tho obviously far smaller serious escalation ones, obviously, but you're exaggerating as well)
1 bug per 10 lines -- not kilolines -- is much closer to the actual "bug rate".
You may be thinking "1 serious security vulnerability bug", but in my experience that too is much closer to 1 per 100 lines to 1 per 1,000 lines, depending on the type of application.
Now, I don't know how many of those bugs were privilege escalations, or could be leveraged to be. I'm not sure that people tried to exploit in the same way back then (or, if they did, if they said anything publicly about the results). Still, even if the majority of those bugs were not exploitable, OSes were much smaller then.
So that's my conclusion of how we got here: We've always been here.
AI helps find vulnerabilities, but the way things are written were already fertile ground.
We developed AIs that can find them.
> > We developed AIs that can find them.
We could already find them before AI.
We already knew they were there, and we already knew how to prevent them in the first place.
See previous discussion: https://news.ycombinator.com/item?id=49705794
Not at this speed and cost.
I'd argue that the resources (i.e. time, money and effort) needed to create the tooling that would have prevented all of these issues would have been orders of magnitude less than the resources that have gone into "AI".
Local privilege exploits have been accepted as being there in Linux ~forever. The past 10-20 years with containers, people have tried to kind of ignore it or pretend it doesn't exist. Nobody with any serious security concerns would ever put untrusted users on the same box, containers or no. Virtual machines are where there is a bit more effort to take this seriously.
You could quibble about complexity and architecture and language but I think at the end of the day it's primarily about incentives. There is not much incentive to make this secure.
I don't know if you would assume all software is more compromised, but you should assume that all software development follows incentives similarly.
What I am really hoping is not that LLMs refine Linux until it is almost acceptable, but that LLMs enable teams to ship useful new operating systems that aren't beholden to decades of macho bullshit.
We were all drunk and manic frankly
The only serious move is kernels adoption of Rust, but for example in Linux this is restricted to 'non core' components.
And while Rust is obviously safer than C, it isn't really 'security focused' for example it still have supply chain attacks, which would be 'impossible' if it used capabilities..
Oh, you strongly recommend rewriting tens of millions of lines of code? Why don't you do it yourself? Just a prompt away, right?
We would also have to burn a trillion tokens a day verifying all the C code the hard way forever going forward. It would be cheaper to rewrite the kernel in Rust, subsystem and driver by driver at a time.
AI doesn't give a strong guarantee that things work though, while Rust gives a much stronger (not absolute) guarantee.
But maybe the way of the future is to auto-generate proofs that the code is memory safe and then just leave things where they are, instead of doing rewrites that require maintainer buy ins. The advantage of C plus proofs would be that the AI generated proofs could live separately so it wouldn't touch workflows of the maintainers.
The only required output they have to process is evidence of vulnerabilities in changes proposed to them. They could outsource even that to contributors.
Humans manually typing out the bytes of source code has been an outdated method, since around a year. It simply doesn't scale as much as those same humans directing agentic teams. So yes, it would be about prompts.
The hard part isn't the technology for the language transition, that one is present today. pgrust shows the way. It's ensuring maintainers are cooperative, i.e. a social problem. Eventually you diverge too much if you do a fork, so it's better done in cooperation with upstream.
Just check how long it took LLVM or GCC to switch off svn for example, and a language switch is much more involved than a VCS switch.
You mean the pgrust which was recently caught for cheating benchmarks and incapable of even performing basic queries outside the benchmark set? Would love to see pgrust debunked by Jepsen like this other AI-generated, written in Rust, "formally verified" scam.
And we're hiring! https://openai.com/careers/search/?q=cyber
(As someone who used to handle nonpublic Linux kernel security reports and is still very interested in this stuff, I’d love to review a small sample of these reports to get a feel for what kinds of issues are being unearthed.)
Also, please don’t stop at LPEs and information disclosures. If you find cases where Linux can be convinced to misbehave or where the implemented semantics don’t make sense, these are good, too. I’ve found plenty of interesting kernel bugs, including some juicy LPEs, and almost none were memory safety errors :)
I understand this work was based on a dated snapshot. Will there be any continuous effort, scanning new PRs as they land, to prevent new weaknesses from being added?
It will be difficult to claim that the problem has been solved without an idea of how big the problem is.
Given that vulnerabilities are almost entirely unknown, you are inherently trying to measure the undetected.
Is there a way to hypothesize how many vulnerabilities there are and them make a prediction on how many of those will be found by a model to get a sense of how accurate the estimate is?
Also there are varying types, its worth getting it tested if you need to do a lot of work with/around it.
E.g., 948 results is a high number to review in a timely manner unless your team is very large or spending several months combing through the results
EDIT: nvm i shouldve kept reading, answered
> At this scale, manually investigating every report is impractical for our research team. We therefore focused on validating findings that a) were reachable by a low-privileged user, b) required no special capabilities, and c) produced measurable evidence of memory corruption.
Multiply that behavior times the number of commercial and mobile and b2b apps over 50+ years, and that's where we are.
There will likely be many more leaks, and hacks, and outages. And then in trying to be proactive to fix old systems, that will cause new proactive outages, and then eventually many things will be better.
But, it's going to take a lot of time and investment, and a lot of (majority) of currently unsupported devices will have to be complete replaced / scrapped / ewaste. On the scale of Nation-State initiatives.
This is across all operating systems, firmware, software and devices of all kinds, etc.
LLMs to security are like a bomb went off. We still have to wait for things to fall over before the dust will even clear enough to see what we have to work with.
The article says "The target virtual machines ran a stock desktop operating system". Are the vulns in the kernel specifically or does this include software running as root?
The only exception I know of is Android. It would be interesting to know how many of these vulnerabilities work there. Can't be zero anyway...
And, well, since you didn't _specifically_ ask for questions about the blog post:
Why are you not quitting your job in protest and screaming at the top of your lungs about it, seeing so closely what these models are capable of that your company is creating? There exists now something that is able to find 948 kernel LPE vulnerabilities. How, exactly, will humans be able to control and contain, say, next year's models for a nonzero amount of time?
Then the question becomes how many vulnerabilities Sashiko didn't catch (If they use/review Sashikos review)
Would be nice if the page could be readable without.
A system that admits THOUSANDS of such vulnerabilities is certainly flawed in several fundamental aspects
So you have to chunk it down to workable pieces, and then if you collect enough data of that process you can look at higher level stats.
Same for every other existing piece of software, btw.
...and then proceeds to never answer a single question. Classic.
There are plenty of design decisions that could fundamentally change things. But they're not being taken. What you're seeing is the revealed preference of this system: Security is important, but not enough to make people incur the cost of adopting major changes in pursuing it.
It's also possible that we'd need even more architectural changes to avoid such bugs.
The idea of a microkernel suddenly becomes more appealing...
And now you have AI that's learned from the other 99% so nothing's changing anytime soon.
The idea that there's a small subset of C programmers who can wrangle the language into obeying them perfectly is obviously, a fantasy.
But if I wanted to learn software design and architecture, I'd probably not even know where to start.
Why is that? Was it just restraint? North Korea didn't show a lot of that to Binance but perhaps they are leaving traditional infrastructure alone because it's harder to profit from.
Actually it seems somewhat remarkable crypto exchanges aren't hacked more often. I guess that's another message from this - somehow people can build secure infra from these apparently broken pieces. Perhaps there's hope.
Why haven’t these been operationalized? NK resorted to social engineering in their latest bitcoin heist despite there been a thousand holes in the system.
To me, this points towards these holes are near impossible for humans and traditional tooling to find and Pandora’s box has been opened, or that these holes are not open to abuse in practice.
I guess the answer is that in most applications you are able to keep most users off the box altogether.
Now, either those are 1k+ unique distinct techniques, or closer to my Sql injection example.
There is no real data attached to the post. And in general, open ai marketing already gained mileage of reputation amongst us.
Most likely kernel is not that vulnerable as they made it sounds.
If only the kernel maintainers considered the thought which first came to your uninformed mind, they would have saved some effort...
And yet the article does not reference any of them. So of the 1000 they found, 400 were already public. And the other 600 we have to believe.
It begs the interesting question on the future of bug bounty programs: when it’s so easy to find and solve issues there’s less incentive to offer a bounty, which in turn leads to less incentive for humans/agents to find and fix security issues..?
As someone not plugged in on the current times with respect to bug bounty programs, curious for peoples take on this / is this already the reality ?
This api over the years has been abused and recently come under a major
fuzzing "attack" through tools like syzbot which decided that it would
attempt to just randomly bind any driver to any type of device, causing
loads of unneeded errors and pointless kernel patches to be generated by
unsuspecting new developers.lol
Convienient to claim "safety" when patches are already public. I await the full writeup on these new techniques so that they can be judged by the broader security community.