Posted by goranmoomin 7 hours ago
Occasionally I see real security researchers on HN complaining that no one takes the disclosure seriously, or that people reply immediately with a cease and desist. But from the receiving end it's just because the spam is unmanageable.
It's hard to spot the stuff that actually matters.
(a) add a new function that does regular expressions searching / matching with a resource checker (eg a timer);
(b) write a local linter that reports an error for any use of the builtin regular expression tools;
(c) fix all the lint warnings;
(d) commit the linter.
e.g. in cPython, until 3.11, there was no support for atomic grouping (roughly translation: "never backtrack inside of this expression"). There is little useful advice a linter can give, if there is no predictable-runtime way to express what you want within a single match step, because you really do want to unwind the stack and check for repeats (just without any of the exponential runtime stuff, please).
[1]: https://meta.stackoverflow.com/questions/328376/why-does-sta...
Ok. Hacking me by changing the input to my Jekyll rather involves being on the other side of the airtight hatch.
Ironically we'd probably need to run Dependabot itself in a mirrored environment since it too has external dependencies we'd probably not want to vet.
I do think external dependencies are among our biggest security threats though. It's so hard to vet them, and compliance basically comes down to "We trust the apache software foundation enough, and pyarrow is vital to our business, so we accept the risks", and then you lock versions and aren't the first to update except for vulnerabilities. Shadow AI is obviously the number one security threat right now, especially in enterprise with people who are very tech savvy. This makes dependencies so much worse though, because now everyone can (if their systems aren't locked down tight) do so many crazy things. Both with the "non-sanctioned" AI but also with the code it can generate for them.
Just updating everything is probably easier than assessing if it's possible to trigger an exploit with the way you use the package.
I do wonder how long it will take before an attack is developed by submitting a semi-genuine vulnerability, shortly followed by a ‘fix’ including malicious code.
Dependencies and supply chain attacks are probably the greatest risk to a lot of software orgs, as they run them across all their environments: Development (with secrets and other valuable artefacts on developer VMs), CI/CD pipelines which may have access tokens to production (and other) environments, and production itself.
Notably even security companies are being impacted by this[0]. The scale of these attacks has amplified quite significantly the past three years, but are not solely exclusive to the javascript ecosystem [1] or even just namesquatting/typosquatting [2].
The resolution is broader security awareness, "onion layered" security controls and implementing simple non-burden inducing processes and policies. Sometimes not updating (what was wrong with the previous version of a dependency if there was no immediate vulnerability or production issue caused by it?) or having a two week cool down for updates (which some supply chain tooling natively supports) can appease some security functions through clear communication of the supply chain risk etc.
If anyone has interest in courses aligned to your org on improving developer and broader engineering management awareness on this, e-mails in my profile :).
[0] - https://socket.dev/blog/ongoing-supply-chain-attack-targets-...
[1] - https://orca.security/resources/blog/hades-pypi-supply-chain...
[2] - https://checkmarx.com/zero-post/python-pypi-supply-chain-att...
The other option is to simply ignore updates and do them on a schedule, e.g. once every 1-2 months.
Even worse, it incentivizes randomly updating dependencies, which is what actually allows supply chain attacks.
> The attackers used a supply chain attack. The attackers accessed the build system belonging to the software company SolarWinds, possibly via SolarWinds's Microsoft Office 365 account, which had also been compromised at some point. SolarWinds was using build management and continuous integration server TeamCity provided by the Czech company JetBrains. In 2021 The New York Times stated that unknown parties apparently embedded malware in JetBrains' software and through this way compromised also SolarWinds.
https://en.wikipedia.org/wiki/2020_United_States_federal_gov...
I don’t know what kind of software you write, how valuable your company’s infrastructure is, etc. But supply chain and insider threat in security/infrastructure is a big topic — that I’m sure they’re concerned about because that’s their area of responsibility.
Even if I’m personally sympathetic to not wanting to deal with the churn of dev dependency updates.
So far as I’m concerned the solution is to isolate everything as much as possible. I’d love to see something on the CVE classification side to also address the signal to noise problem but I don’t see it happening.
If I could filter out DoS CVEs‚ I would.
And your ISO etc certificates make this CVE mandatory priority 1 action point
I have already avoided having to evaluate whether I am affected by some issue because I added patches at startup that crash before certain unused-yet-installed modules are to be loaded. Also, for those Python packages that still have a pure version that defers to stdlib and a separate muh-performance binary option with statically linked dependencies, I can generally just install the former and skip the version bumps for dependencies. The performance advantage may be negligible or negative outside of benchmarking 100k calls.. of code actually called 11 times a day, on a non-critical path.
(besides cve, nist, et al drop in criteria) searching for an indepth analisys, you find one million (after scrolling the Ai summary) results that are either copy-pastrle or Ai rewording of the cve announcement.
...and don't get me started on the proofs that stop after smelling the attack vector. you can't evaluate if your setup is DoSable at most or full remote shell.
there's still tons of good analisys and reports. but the noise....
Sure - modern AI can figure that out, but I bet in a vast majority of cases they won't.
Again, the alternative might be shutting down the program entirely.
They already like blurring the lines by rejecting reports that have clear reproduction scripts, videos, demonstrable (but not critical) impact. They'll close it as "not a bug" but then also forbid disclosure and stonewall mediation requests. Reports are supposed to be kept private until the issue is fixed but the system gets abused to cover up issues long after they've been fixed.
In some cases I strongly suspect it's to evade liability for financial damages that their customers might've suffered. Platform mediation always takes their side and if you want to do what's right, you will get banned.
First question: Do you keep money for shit reports?
Well no, you have to pay it back like credit card validation. There is no pain for posting shit report just inconvenience. There is no legal way where you can keep the money.
But do you know how many people are doing things that are even dumber right this very minute? I don't know either, but I'm sure it's larger than either of us would like to admit.
I still check the spam folder for legitimate emails, but so far there haven't been any false positives.
Also people ironically just DGAF that much. The last actual bad exploit was log4shell in java, which given how it was introduced (i.e someone purposefully at Apache made it so a log statement can execute code, and nobody questioned it before pushing it to prod), should have been the signal for everyone to completely remove all Apache libraries from their services, but yet all the software is still being used.
We’ve always been able to find heaps, we’ve just never had the right structures to put in the effort and renumerate people for looking (even if they don’t find anything).
Further, the fact that bugs are so easy to find by LLMs means there is strong incentives to find ways to minimize creating bugs in the first place. That could be new or better languages, less 3rd party dependencies, more vetted code, better linters, better fuzzers, whatever. The point the new reality of bugs being easy to find will, actually must, lead to less bugs eventually because the world can't function with easy to find bugs.
Then a year or two later they'd repeat the operation and they'd find about the same amount of same types of bugs. In many occasions in code that had been in place in the previous round and had remained essentially untouched.
Paraphrasing what the Gruqg has quipped - a large piece of software has infinity bugs. Infinity minus N is still infinity.
0: Discovery rate with regards to the time spent looking for bugs. LLM-powered bug hunting has amped up the speed with which code bases can be investigated.
I find that often bugs will be created when using an LLM, like others have said. Saying that this can then be fixed by identifying all the bugs created by an LLM with an LLM doesnt guarantee another bug is not introduced when the LLM is addressing the initial problem.
Also, what if the LLM has a blind spot. They certainly also could be incapable of finding or fixing a bug. They dont pass any benchmark at 100% right now. Also also, guaranteeing there are no bugs in your code is like saying you have 100% test coverage, all of the tests pass, and they are written perfectly. Saying that you can simply identify and fix the bugs also assumes there is enough time and energy to find all of the bugs that exist within a project and then to address them. Even LLMs use time and energy. In a sufficiently complex system that is certainly wishful thinking.
Considering the size and complexity of a lot of modern software (like web browsers, 3d modelling software, game engines, etc.) software is just too complex to not have bugs even when created and managed by LLMs.
There will continue to be bugs in code and we will simply have to live with the fact that LLMs make it easier to exploit computer systems. I mean consider a hardware bug like Spectre [0]. If bugs like this become easier to find does that mean our existing hardware will just become obsolete more quickly? that type of problem can be addressed, but at quite a high cost.
Not sure what all of this means for the future.
0. https://en.wikipedia.org/wiki/Spectre_%28security_vulnerabil...
Read what wrote, I didn't say your program will be bug free. I said, if the LLM can trivally find the bug it will. If it can't then we're at worst, back to the state of before LLMs could find bugs, but likely much better since we fixed so many of them
So, the fact that LLMs can trivially find bugs is enough to get the bugs fixed.
You, and several others, seemed to think I was saying LLMs would fix all the bugs. I never said that. I said they'd help. Finding them is help. Writing a possible fix is often help. Writing a possible fix and seeing if they can detect a bug after the fix is applied is also help. Automating the entire things and letting LLMs fix them without review is likely not help.
I agree that LLMs make finding bugs take far less time and energy. I also agree that this should mean in the long run there are less trivial to find bugs IF everyone adopts the usage of LLMs while writing and reviewing code.
It does also seem possible that LLMs are better at finding bugs than fixing them.
I think we're at the point that the best LLMs can indeed write software that's far more secure than your average programmer. Partly because the average is so terrible.
Are you making a counterpoint that the reports are so good and must all be addresses, but the problem is "llm finding all the bugs" so fast us poor slow humana cannot keep up?
because if so, i suggest you write a new article.
Hopefully at the end of this decade, a ton of software practices have been overhauled to eliminate classes of problems. Memory-safe language use is a great start - but it’d be great to see innovation in checking for TOCTOU problems, improper/missing authn & authz, and many others.
This is an engineering problem. It won’t be solved by models that “only do dumb shit 1/10th as often, only 0.01% of the time now not 0.1%!” It won’t be solved by adding more models to do even more double-checking before and after the work. It won’t be solved by hoping humans catch it in review. It isn’t solvable by adding outer loops of any sort - though we may get close. To truly solve this will take serious CS research.
Personally i have some doubts, a lot of research has gone into the idea without much to show for it, but its a very reasonable research area.
Part of what the research shows is that correctness-by-proof has a cost in developer effort.
If there really is a vulnerability-apocalypse due to AI, and it's not just a different flavour of AI hype, the cost of having insecure software will rise to the point that the cost of dealing with insecure or incorrect code at time of creation becomes less than the cost of ignoring it until it blows up.
I doubt it'll rise so much that we'll want to face the cost of behaviour proofs for much code at all, but it's quite possible it'll rise enough that we want to do things like prove that indices are in bounds, at compile time, so vector accesses can skip checks without compromising safety.
But yeah, writing specs is usually harder than reviewing the code 4 times :)
See WPA2 KRACK, you could've had a formally verified WPA2 implementation and it still would've been exploitable because the flaw was the specification itself.
> But give it 1-3 months and the open models will catch up.
I wish that this would stopped being thrown around, what is this timeline based on? How good is your open model from between March and May?
Also, having read "Gödel, Escher, Bach" I know that the hare never catches up with the turtle.
> A requirement for staying sane while working in public as an open source maintainer is realizing that every issue, PR, and piece of feedback is a present, not an obligation. You can accept it, ignore it, and use it partially or not at all.
> Except…
> For years, as lead of the Go Security team at the time, I’ve told new team members that it doesn’t apply to vulnerability reports. No, vulnerability reports are special. Security researchers are doing us a favor by reporting things confidentially instead of doing full disclosure, so we owe them something, which is not true of regular issues opened on the issue tracker.
[...]
> It’s 2026 and none of the premises are true anymore.
I respectfully disagree.
The premise is absolutely still true: if someone discovers a critical, exploitable vulnerability in your software, the impact and tradeoffs are exactly the same as they were before LLMs started finding bugs. There are just more of them now, so they're easier to come by.
But that won't last forever, either. As LLMs find increasingly difficult-to-find vulnerabilities, there will be fewer of them to report. This is just chugging through the backlog.
All of that said, I don't think finding vulnerabilities has really been the difficult security problem for most companies (or open source projects). The difficult problem is dedicating resources to fixing those vulnerabilities instead of building software, products, and/or infrastructure that people want. That problem is absolutely still here today, but I'm optimistic that agentic security developers will be able to take the burden off of development teams in the near future.
For tokens, of course.
I think your logic is partly correct but the fact that the same LLMs are allowing an exponential increase in insecure code generated is a counterbalancing point. I do not think this phenomena will slow down.
That is not my experience at all. People will continue to high-volume spam intended behaviour as if it is a bug.
There will be fewer reports that matter as you fix things - but the volume of reports will either stay steady or go up. Making it harder to even notice the ones that matter.
I think the point is those issues are now easily discoverable and are nearly public because of it.
> The security researchers are not special, the insight and confidentiality are
vs
> The bottleneck now is not finding potential issues but assessing which ones are real. Unless there’s already a trust relationship, external researchers can’t meaningfully contribute
My take-away from this is that the researchers were special all along and you should probably be building a trust relationship with them.
Despite what I want to believe about tech being a meritocracy, the reality is that trust plays an extremely important role and without it we risk a collapse of our open source software ecosystem.
One of my biggest criticisms of AI is the trust vacuum within which it operates