Top
Best
New

Posted by ymir_e 10 hours ago

SQLite Critical CVEs or LLM Slop?(research.jfrog.com)
678 points | 325 commentspage 2
trashb 9 hours ago|
Could be just me but I would be quite careful running these PoC's, especially since the CVE's mostly seem nonsensical. I suppose these CVE's could be targeting some kind of PoC execution instead of actual bug reporting? (I didn't look at the code in depth).

Why is the repo even mixing CVE's for "schreibfaul1 ESP32-audioI2S" and "SQLite"? Is mixing CVE's for different products in one repo common practice?

inigyou 7 hours ago|
There is no specific practice for a repo full of bug reports. Having a repo full of bug reports is already unusual. Normally you report a bug against the software that has a bug, not by creating a file in your own repo.
_flux 7 hours ago||
I've created bug reports to open source projects that have related files in my repo. Then I can just list the reproduction instructions like:

1) arrange xxx, such that yyy.. or git clone this repo where this is set up. 2) ..

I thought that might have been quite helpful not just to the person I'm sending the bug report to, but also to myself when I need to evaluate if the bug has been fixed.

umarcyber 5 hours ago||
It really is a shame that funding cuts at NIST has led directly to this. I don't know if there is an incentive for a non-profit to come out of the woods to do the validation and enrichment process, but boy do we need it.

Like gortok has said, LLMs are just probabilistic text-prediction engines. They cannot automate the vulnerability validation lifecycle because they cannot account for improbable edge cases. They may not even get the version of the software right...

wildfireday2 5 hours ago|
Funding cuts at NIST did not lead directly to this. You must be new here or haven’t spent a lot of time directly interfaced with NIST on cybersecurity in decades past.

And NIST’s role with CVE’s has always been purely ministerial/clerical. It has never been their job to do even a cursory investigation into the vulnerability itself.

jrochkind1 5 hours ago||
OP suggests something at NIST changed in 2024, where they stopped doing as much verification as they did before.

To be sure, the suggestion is not funding cuts, but an increase in workload with same funding.

Here is the 2024 NIST announcement OP links to: https://nvd.nist.gov/general/news/nvd-program-transition-ann...

> Currently, we are prioritizing analysis of the most significant vulnerabilities. In addition, we are working with our agency partners to bring on more support for analyzing vulnerabilities and have reassigned additional NIST staff to this task as well.

The OP described this as "NIST effectively hit pause on deep analysis. "

It does sound like they stopped doing something they used to do in 2024. I personally have definitely not spent a lot of time directly interfaced with NIST on cybersecurity in decades past, I know nothing about it, just what I read in OP.

Are you saying the OP was wrong to call what NIST used to do "deep analysis", and/or that the thing NIST stopped doing was "purely ministerial/clerical" in a way that it would not have caught fake reports anyway, contradicting the OP? Or other?

Again, to be sure, the OP's suggestion was not that this was caused by NIST funding cuts, but by "a massive surge in vulnerability reports,"

mbauman 4 hours ago||
The main point is NIST is _downstream_ of CVE issuance. Yes, they can — and still do — add disputed/rejected tags to CVEs, but in many cases by then it's already "too late." The CVE has an ID and a lifespan of its own.

NIST does not and did-not/cannot/never-has unilaterally "retracted" CVEs or prevented their issuance.

But yes, NIST's situation is not good for the world. The services they provide are hugely valuable.

jrochkind1 2 hours ago||
I only know about this what I learned from OP and you guys.

It sounds like you guys think OP was mistaken, whatever analysis NIST was doing that they reduced in 2024 would not have prevented this anyway?

Legit question, I'm trying to understand!

OP says:

> Hit by a massive surge in vulnerability reports, NIST effectively hit pause on deep analysis. CISA and other Authorized Data Publishers (ADPs) tried to step in with their own enrichment efforts, but the global pipeline is now fragmented and drowning in a massive backlog. Because no step in today's system actually requires a proof-of-concept or bug reproduction, a plausible-sounding fake advisory can slide right through the pipeline and end up in GHSA, downstream databases, and enterprise scanners.

Do i correctly understand you are saying you think they've mistunderstood the diagnosis of what changed, the 2024 change to NIST didn't actually make it any more likely for a fake advisory to go through pipeline?

bilater 5 hours ago||
This isn't great but a negative side effect of an otherwise positive change. I know a lot of comments will try to make this into a gotcha but its really not. Decreased friction leads to misuse. Just like before we'll figure out ways (reputation, light filter pass) to deal with it.
jrochkind1 5 hours ago||
The federal government ought to be funding NIST sufficiently to actually do sufficient analysis. Do we care about funding civil "cyber" defense or not?

(Obviously the answer is not, we only care about funding offensive capabiltiies).

Plutoberth 4 hours ago||
This is obviously impractical. With the volume of bug reports that are generated and such a wide breadth of software no single agency will be able to handle all reports. And honestly? It doesn't matter, even today. CVE should serve as a reasonably deduped identifier of specific vulnerabilities. It was never interesting and practical to care about 100% of vulnerabilities in a specific deployment, and it's not interesting today.
inigyou 5 hours ago||
The federal government defunded the CVE program last year, I think, because it was woke.
taf2 7 hours ago||
Llm are great at things that are repeatable. Otherwise they are just like us - great at believable bullshit… a sign of true intelligence
firesteelrain 9 hours ago||
> This incident demonstrates a systemic issue with automated vulnerability ingestion

But they make X-Ray which does automated vulnerability indexing and matching dependencies to CVEs.

dvh 10 hours ago||
You're absolutely right...
mike_hock 9 hours ago|
It's not actually vulnerable.

You're absolutely right. I made a critical error. It's NOT vulnerable.

It' actually vulnerable.

You're absolutely right. I made a critical error. It IS vulnerable.

It's not actually vulnerable.

You're absolutely right. I made a critical error. It's NOT vulnerable.

inigyou 7 hours ago||
Hilariously, this comment tripped the LLM writing filter and was auto-deaded. I vouched for it.
minraws 6 hours ago||
I think this will give a lot of ammunition for people on both sides, but the thing is we have made software so complex, ofc a lot of it is inherent, that we can't even tell if the vulnerability is hallucinated or not.

I find it getting closer and closer to the nonsensical 50 page mathematical proofs built with computer assited tools, that been driving reason for abstract proof systems so that we can break down such complicated examples into something simpler that we can feed into a system to help us verify.

I feel like for some reason we have built the output machine before we could even build the actual output verification machine. And aren't ready to accept that we should all move towards building some kind of system to better help verify the "truth" produced by these AI systems.

I am now very convinced that human built and proven to work verification systems for LLM outputs are more important than ever, but I just don't see any company or team working towards it.

Trillion dollar systems shipping to production code that is hardly understood by any one without any verification what really could go wrong. Honestly I really truly hope nothing goes wrong.

To note I am not anti-AI but I want to be able to have my LLM outputs be verified by more than just me under a tighter and tighter deadlines, accepting worse and worse slop just because it would less cost to me to accept the PR and then fix it in prod.

LLMs aren't sustainable software development, they are delusional slop factories if we aren't able to actually make sure the outputs have any amount of correctness.

With humans it worked because all 10 of us could in some ways verify each others outputs, and had trust in the fact that our timelines included the time for reviews.

sabot90260 9 hours ago||
A critical rating on a vuln that doesn't exist is wild. The triage cost still lands on the maintainers.
kasperset 9 hours ago|
As a non-expert in this field, I feel we have introduced more “stochasticity”. Before, We had to deal with some human error/ misunderstanding for most part but now we have to deal with power of condensed human expertise along with homogenized flaws.
More comments...