Posted by Bender 1 day ago
Our current quota is, ca. 20% of AI-found "critical" issues are in fact critical. 3-5% of reported "high" are also critical (i.e. unfortunately it's not possible to rely on AI being excessively alarmist, it does under-classify some issues.) This is with erring towards higher severity though; we didn't get anything that'd e.g. make HN front page.
Simultaneously, ca. 35% of reported "critical" are not even actual security issues. (May be biased due to the nature of our project, AI seems to have issues understanding security boundaries.)
Possible some of this is due to "they're prompting it wrong". Hard to tell since we don't have access ourselves.
I've not seen major differences between models & vendors, but we mostly get reports from Anthropic. (We currently just treat them as functionally interchangeable, which is a data point in itself.)
> The total run cost $1,213,430 in API credits, or $1,279 per LPE exploit.
On a sidenote: I'm guesstimating the total cost on maintenance side will be 1x…10x of this. (Mostly humans to review fixes. And this isn't made cheaper by AI in my experience; to get a reasonable level of assurance regarding larger design aspects & co, you just end up spending a similar amount in tokens as the involved humans' time would cost.)
What percentage of ALL returned AI security bugs are critical?
Currently ca. 2%. Not sure this is a useful measure though, all of the batches had a whole bunch of low severity stuff (correctly flagged as such) that technically is indeed a security issue but practically irrelevant.
> In a run in mid-October 2026, our pipeline using an upcoming cyber model dynamically reproduced 1,724 vulnerabilities and generated PoCs for 291 information leak vulnerabilities and 948 working local privilege escalation (LPE) exploits for a commonly used Linux distribution. The total run cost $1,213,430 in API credits, or $1,279 per LPE exploit.
Not saying LLMs aren't pushing hard in security research right now, just that the tangibles seems to come largely from a knowledgeable person pushing an agent towards the find and not some autonomous process.
I imagine the people who are committed to keeping LTS versions of the kernel patched are having a terrible time right now. I guess at least AI tools make the patching part easier.
You'd hope so because if it turns out there is an asymmetry between "finding the hole" and "filling the hole" in terms of model capability that's a certified Bad Time(TM) for everyone.
Hell even if there isn't, We still have decades of cruft to fix and we already knew before LLM's that in general defending is harder than attacking, they only have to be lucky once, we have to be lucky every time.
It's going to be a rough few years.
"Oh, they matter, but we gotta push out fixes for the 4000 actively exploited holes we have already"
Capacitor based memory will be, and will remain, vulnerable to this. Capacitors being charged and discharged will generate radio waves and radio waves will change the state of other capacitors. There is no avoiding this, there is only minimizing it, and at the scales we're talking about, there's only minimizing it somewhat.
There are probably a lot of patterns we haven't seen yet. Like memory write patterns that change other memory by having the waves interfere.
You can only make it more difficult by thinking better about memory layout, and change it a bit by thinking about the physical layout of memory cells. But you're only changing the patterns. It's not possible to prevent it. You cannot fix this. Not now, not yesterday, not in 100 years.
It's currently too difficult for human to think about (even though people are slowly publishing about yet another system being vulnerable to rowhammer style attacks that work a little different from the existing ones)
So why isn't it "game over", at least for the web, for phones and apps, for ...? That bug isn't solved an nothing short of switching everything to SRAM, at incredible cost, will fix it, not just in legacy systems, but new systems too, now and in the future.
Unfortunately, we live in a world where the few percentage points on the production cost of DRAM needed to completely mitigate rowhammer is sufficient to ensure that defective DRAM continues to be sold well after the industry became aware of the problem. This is one of the instances where a class action lawsuit might actually do some good.
Which they're not doing, in fact they're doing the opposite since what you're suggesting is both expensive and makes things (a lot slower). Oh and while I'm complaining: thousands of people are investigating how to place DDR much closer to other logic that also emits radio waves.
Oh ... and even if you did move the capacitors apart, that would make it more difficult. But by using more cells to hammer you can still make it fail. That would even be counterbalanced because the frequency needed for triggering rowhammer would go down as well, so it'd be a more forgiving environment for exploits.
1. increased the frequency of the interface (mostly in the phy / logic part of a DRAM)
2. increased the number of banks within a single die
3. split the 64 bit channel on a single DIMM into 2 x independent 32 bit channels
(1) is the biggest problem for DRAM performance scaling, and that's because the legacy of DIMMs is awful from a signal integrity perspective. As signals go above 10Gbps/pin, the length of stubs begin to impact signal integrity, and traditional DIMM sockets have pins going all the way through a motherboard. This is why LPCAMM and LPCAMM2 exist: to improve signal integrity, but LPCAMM2 is not yet widely adopted on desktops and servers.
FYI: DDR5 wasted logic on DRAM for latency increasing crappy on-die ECC (Error Checking and Correcting) because DRAM vendors are unwilling to make the correct engineering trade-offs to build reliable DRAM. As DRAM processes are not exactly great for logic, this makes little sense to me as I am a firm believer that DRAM should stick to being memory and let the memory controller and CPU / GPU vendor implement ECC instead of allowing DRAM to lie about how reliable it is.
If you really want to improve DRAM performance, do what the Chinese fabs are doing: build layers. Making use of the third dimension reduces wire length, lets DRAM manufacturers use larger capacitors resulting in higher reliability and reduced refresh frequency. Layers have been used in flash production for more than a decade now, and due to trade restrictions on EUV fabs, China has been pursuing 3D scaling aggressively. There is no reason that DRAM stacking should be limited to HBM attached to specialty SKUs of FPGAs and GPUs.
Speaking of HBM, HBM takes the approach of using a very wide bus (1024+ bits) at a much lower clock speed tp achieve high bandwidths (which makes maintaining signal integrity a lot easier). With LPCAMM style DRAM modules, increasing the width of DRAM busses becomes much less daunting compared to DIMM style PCB modules.
So, no, I don't think that increasing cell sizes in DRAM is a major pain for DRAM manufacturers. I think it's time that the customers speak up and refuse to accept marginal failure prone DRAM chips from vendors that value profit over a solidly engineered product.
[Side note: several papers have mentioned that not all DRAM manufacturers are vulnerable to rowhammer attacks.]
The physics makes the rowhammer impossible to fix in practice, but you can add various mitigations: scrambling, ECC, detection and response, etc... On the application level, you can also make it so that it is not possible to do a rowhammer in the first place. Fix everything that can be fixed and mitigate what can't be fixed.
And there is another security issue that can't be fixed, and is much worse than rowhammer: people. But we can't remove people, just like we can't remove DRAM for the performance level we need, so we mitigate the risk. It is not game over because we have other layers of security.
To go back with the human analogy. I have a server in a datacenter, there is a weakness in that someone can go to the machine physically and do nasty stuff, including hitting it with a literal hammer. This is a risk, but I can't do anything about it. Someone has to do the maintenance. But it can be mitigated with access control to the datacenter, among other things.
If the door to the server room can be opened with a toothpick, that's a vulnerability and it has to be fixed, because to "real" solution of not letting anyone in is not viable. I didn't chose to let people in the server room, it is just impossible not to.
If software improved to the degree that hardware and physics-level exploits were the easier, we would be in a world 100x more secure than what we have now.
The real question you should be asking is why is it not “game over” right now given that most people would only hope in their wildest dreams that they would be so lucky that worrying about Rowhammer and the like would be a good use of time compared to the other much larger gaping software holes.
The vibe coded nature of letting LLMs write patches without understanding the impacts, could invite bandaids on top of bandaids.