Posted by goranmoomin 8 hours ago
I wonder what the metrics are. Also, not "anyone", just the affordable.
> Ultimately, it all stems from our responsibility to our users. The security researchers are not special, the insight and confidentiality are, and we need them to keep our users safe. Ignoring a security report communicates you don’t care about users’ security, and it’s rightly a reason for shame.
100%. This was always true and I still think it is. LLMs don't change anything. At most they shift the balance and force a temporary compromise.
> LLMs are as good as almost any security researcher
This statement is extremely dependent on the definition of a security researcher. It might hold if you consider anyone with a HackerOne account, but if you restrict the definition to people who actually put in some effort, it's just not true. LLMs can find some real vulnerabilities, yes, but they also spew unprecedented volumes of garbage that an expert can immediately recognize as such.
> The insight is not scarce and precious anymore. The bottleneck now is not finding potential issues but assessing which ones are real.
Assessing which ones are real should be part of the insight. Real researchers will not submit 150 pages of spam, and three real bugs hidden in 150 pages of spam are not insight. In most cases a researcher will spend significant effort on triage before submitting anything, and an LLM still cannot do that reliably.
> Confidentiality, embargoes, and coordination also don’t matter nearly as much as they used to.
I'd argue these now matter more: the one thing LLMs do seem to do fairly well is figure out specific things based on sufficient information and a scope that's limited enough. So a plain commit containing a security fix is now much easier and cheaper to turn into an exploit than it was before.
> The years of vulnerability reports being special might be over, as weird and uncomfortable as that feels.
I'd hope not. Bug bounties might be over unless someone can figure out the spam problem, but disclosure programs that don't offer monetary incentives are probably just going through a tough period that will eventually calm down as the operators of LLMs realize the costs and do the math.
Unreliable reports have always been an issue and will remain one, LLMs or no. When it gets worse, like in the current influx of LLM-generated reports, the focus should be on identifying reliable researchers, building relationships, and providing guidance on how they can make the reports easier to triage.
Researchers are not special, but the insight they can provide totally is. LLMs might force everyone to make better use of that insight, instead of just consuming bug reports and drowning in triage.
[1] https://groups.google.com/g/golang-announce/search?q=juho
> We're keeping our vulnerability disclosure program open - because even though they are rare, the genuine critical reports we receive, in amongst the noise, are still highly valuable. I don't think we're at the stage yet where finding those issues is a purely mechanical process; persistent, imaginative researchers still make a contribution to the process by finding things that LLMs by themselves, so far, haven't.
[1] https://www.linkedin.com/feed/update/urn:li:activity:7475447...
One flipside to this is that, because many of these bugs are "shallow" to LLMs, it's actually easier than ever to moderate the worst participants in your vulnerability program -- if someone sends you slop, you can just ban them and wait for the next, better orchestrated LLM to send you a better report for the same vulnerability.
The _demonstration_ of security impact through vulnerability reports was special. The automation of “demonstration of impact” with AI isn’t that at all. The last mile is human and always was. This isn’t to say it won’t change in the future, but that’s a fact of where we are now.
Vulnerability reports aren’t special anymore. They never were. It was the impact, the demonstration, the communication that was special.
When you realize that this is being written from the perspective of someone who does vulnerability reporting in a professional capacity, you’ll connect the dots. We took care to be kind and succinct because for many of us, we learned our skills from being on the development side of things first.
Vulnerability reports aren’t special anymore. The only ones that felt special were the ones with human touch, the ones doing their job as an adversarial thinker, and taking the care to understand that net positive outcomes require coordination even if both parties don’t see eye to eye.
Nothing has changed. It never was. You’re just inundated with AI slop; which as a practitioner who uses AI regularly I can say with absolute confidence. The end result is the same, the volume is increased, but the special thing was never the report itself.
Finding a vulnerability was always the easy but high toil part. It was the care to communicate succinctly and be invested in the outcome that was special.
Godspeed.
I’ve been screaming this from the rooftops. Impact is what was always important. No one is going to take down prod to do an emergency patch on an RCE that COULD NEVER ACTUALLY BE EXPLOITED.
I feel like we’re witnessing the result of multiple roles suddenly becoming security aware but not having the background or understanding to make any sense of it.
If you aren't careful, that is how you get this security theater.
This of course made vulnerability researchers seethe worse than aggrieved Redditors.
It turns out he was right all along.
The author also gets it wrong by assuming that regular bug reporters are not "providing a service". They are.
When I wrote up a bug report, I made sure it's thorough with detailed steps to reproduce. It takes a lot of time and I've done it professionally for projects you've absolutely heard of.
Having said that, getting them ignored repeatedly and — even worse — having my detailed PRs rejected, sometimes within minutes, as if I'm some ignorant luser is why I don't do it anymore. My time is more valuable than your hubris.
A lot of open source developers have their heads so far up their own asses they forgot that it takes a community for projects to be successful.
I filed a report, they marked it as 'informative' and thanked me, recommended I keep looking for more vulnerabilities, but no payment at all; they said I had to be able to demonstrate major disruption of service... Which I presume is illegal. I literally showed them all the ingredients of the attack, the exact curl commands, payloads, the exact response delay could be easily be verified; you could see the server response slowing down proportional to the degree of nesting in the payload. I could execute it without authentication too; so it was essentially certain that the attack could be scaled but they made it impossible to get a reward.
The hardest part was writing the report which took several hours.
So yeah, 30 minutes of looking for a vulnerability, no prior experience in security research, first project I looked into on Hacker One, ever... A company in crypto sector which is a major target of hackers and takes security relatively seriously.
Imagine how insecure most software is! Imagine how bad most vibe-coded software is especially! Companies might as well run their servers directly inside Kim Jong Un's data center in North Korea.
North Korean hackers probably have a dashboard which shows more detailed and accurate platform analytics than what the founders of the company can see.