Top
Best
New

Posted by todsacerdoti 6 hours ago

Turn Dependabot Off(words.filippo.io)
309 points | 79 comments
nfm 5 hours ago|
The number of ReDoS vulnerabilities we see in Dependabot alerts for NPM packages we’re only using in client code is absurd. I’d love a fix for this that was aware of whether the package is running on our backend or not. Client side ReDoS is not relevant to us at all.
staticassertion 4 hours ago||
TBH I Think that DoS needs to stop being considered a vulnerability. It's an availability concern, and availability, despite being a part of CIA, is really more of a principle for security rather than the domain of security. In practice, availability is far better categorized as an operational or engineering concern than a security concern and it does far, far more harm to categorize DoS as a security conern than it does to help.

It's just a silly historical artifact that we treat DoS as special, imo.

jpollock 4 hours ago|||
The severity of the DoS depends on the system being attacked, and how it is configured to behave on failure.

If the system is configured to "fail open", and it's something validating access (say anti-fraud), then the DoS becomes a fraud hole and profitable to exploit. Once discovered, this runs away _really_ quickly.

Treating DoS as affecting availability converts the issue into a "do I want to spend $X from a shakedown, or $Y to avoid being shaken down in the first place?"

Then, "what happens when people find out I pay out on shakedowns?"

staticassertion 4 hours ago|||
If the system "fails open" then it's not a DoS, it's a privilege escalation. What you're describing here is just a matter of threat modeling, which is up to you to perform and not a matter for CVEs. CVEs are local properties, and DoS does not deserve to be a local property that we issue CVEs for.
michaelt 3 hours ago|||
> If the system is configured to "fail open", and it's something validating access (say anti-fraud),

The problem here isn't the DoS, it's the fail open design.

jpollock 3 hours ago||
If the majority of your customers are good, failing closed will cost more than the fraud during the anti-fraud system's downtime.
Lichtso 2 hours ago||||
> I Think that DoS needs to stop being considered a vulnerability

Strongly disagree. While it might not matter much in some / even many domains, it absolutely can be mission critical. Examples are: Guidance and control systems in vehicles and airplanes, industrial processes which need to run uninterrupted, critical infrastructure and medicine / health care.

technion 48 minutes ago|||
These redos vulnerabilities always come down to "requires a user input of unbounded length to be passed to a vulnerable regex in JavaScript ". If someone is building a hard real time air plane guidance system they are already not doing this.

I can produce a web server that prints hello world and if you send it enough traffic it will crash. If can put user input into a regex and the response time might go up by 1ms and noone will say its suddenly a valid cve.

Then someone will demonstrate that with a 1mb input string it takes 4ms to respond and claim they've learnt a cve for it. I disagree. If you simply use Web pack youve probably seen a dozen of these where the vulnerable input was inside the Web pack.config.json file. The whole category should go in the bin.

staticassertion 2 hours ago|||
I think this is just sort of the wrong framing. Yes, a plane having a DoS is a critical failure. But it's critical at the level where you're considering broader scopes than just the impact of a local bug. I don't think this framing makes any sense for the CVE system. If you're building a plane, who cares about DoS being a CVE? You're way past CVEs. When you're in "DoS is a security/ major boundary" then you're already at the point where CVSS etc are totally irrelevant.

CVEs are helpful for describing the local property of a vulnerability. DOS just isn't interesting in that regard because it's only a security property if you have a very specific threat model, and your threat model isn't that localized (because it's your threat model). That's totally different from RCE, which is virtually always a security property regardless of threat model (unless your system is, say, "aws lambda" where that's the whole point). It's just a total reversal.

bawolff 3 hours ago||||
The real problem is that we treat vulnerabilities as binary without nuance. Whether a security vulnerability is an issue depends on context. This comes up a lot for DoS (and especially ReDoS) as it is comparatively rare for it to be real, but it can happen for any vulnerability type.
staticassertion 3 hours ago||
I don't really agree. Maybe I do, but I probably have mixed feelings about that at least.

DoS is distinct because it's only considered a "security" issue due to arbitrary conversations that happened decades ago. There's simply not a good justification today for it. If you care about DoS, you care about almost every bug, and this is something for your team to consider for availability.

That is distinct from, say, remote code execution, which not only encompasses DoS but is radically more powerful. I think it's entirely reasonable to say "RCE is wroth calling out as a particularly powerful capability".

I suppose I would put it this way. An API has various guarantees. Some of those guarantees are on "won't crash", or "terminates eventually", but that's actually insanely uncommon and not standard, therefor DoS is sort of pointless. Some of those guarantees are "won't let unauthorized users log in" or "won't give arbitrary code execution", which are guarantees we kind of just want to take for granted because they're so insanely important to the vast majority of users.

I kinda reject the framing that it's impossible to categorize security vulnerabilities broadly without extremely specific threat models, I just think that that's the case for DoS.

There are other issues like "is it real" ie: "is this even exploitable?" and there's perhaps some nuance, and there's issues like "this isn't reachable from my code", etc. But I do think DoS doesn't fall into the nuanced position, it's just flatly an outdated concept.

bawolff 1 hour ago||
I am kind of sympathetic to that view. In practise i do find most DoS vulns to be noise or at least fundamentally different from other security bugs because worst case you get attacked, have some downtime, and fix it. You dont have to worry about persistence or data leaks.

But at the same time i don't know. Pre-cloudflare bringing cheap ddos mitigation to the masses, i suspect most website operators would have preferred to be subject to an xss attack over a DoS. At least xss has a viable fix path (of course volumetric dos is a different beast than cve type dos vulns)

akerl_ 2 hours ago|||
Maybe we should start issuing CVEs for all bugs that might negatively impact the security of a system.
ranger207 1 hour ago||
The Linux kernel approach
Twirrim 1 hour ago|||
I've been fighting with an AI code review tool about similar issues.

That and it can't understand that a tool that runs as the user on their laptop really doesn't need to sanitise the inputs when it's generating a command. If the user wanted to execute the command they could without having to obfuscate it sufficient to get through the tool. Nope, gotta waste everyone's time running sanitisation methods. Or just ignore the stupid code review tool.

junon 4 hours ago|||
I maintain `debug` and the number of nonsense ReDoS vulnerability reports I get (including some with CVEs filed with high CVSS scores, without ever disclosing to me) has made me want to completely pull back from the JS world.
adverbly 4 hours ago|||
Seriously!

We also suffer from this. Although in some cases it's due to a Dev dependency. It's crazy how much noise it adds specifically from ReDoS...

robszumski 4 hours ago||
Totally hear you on the noise…but we should want to auto-merge vs ignore, no? Given the right tooling of course.
kbolino 3 minutes ago|||
You need to weigh the generally low-severity and/or situationally exploitable bugs that this policy might fix in your code against the high-severity intentionally exploitable vulnerabilities that this policy eventually will introduce to your code.

Supply-chain attacks are already a big threat and they are only going to get bigger.

UqWBcuFx6NV4r 3 hours ago||||
We could just skip some steps and I could send you a zip file of malware for you to install on your infra directly if you’d like.
dotancohen 3 hours ago|||
No
candiddevmike 4 hours ago||
Using something like npm-better-audit in your linting/CI allows you exclude devDependencies which cut down a ton of noise for us. IDGAF about vite server vulnerabilities.
ImJasonH 5 hours ago||
Govulncheck is one of the Go ecosystem's best features, and that's saying something!

I made a GitHub action that alerts if a PR adds a vulnerable call, which I think pairs nicely with the advice to only actually fix vulnerable calls.

https://github.com/imjasonh/govulncheck-action

You can also just run the stock tool in your GHA, but I liked being able to get annotations and comments in the PR.

Incidentally, the repo has dependabot enabled with auto-merge for those PRs, which is IMO the best you can do for JS codebases.

apitman 4 hours ago||
I find dependabot very useful. It's drives me insane and reminds me of the importance of keeping dependencies to an absolute minimum.
keyle 19 minutes ago|
I agree, I don't have a ton of projects out there though.
tracker1 4 hours ago||
I kind of wish Dependabot was just another tab you can see when you have contributor access for a repository. The emails are annoying and I mostly filter, but I also don't want a bunch of stale PRs sitting around either... I mean it's useful, but would prefer if it was limited to just the instances where I want to work on these kinds of issues for a couple hours across a few repositories.
BHSPitMonkey 4 hours ago||
You can add a dependabot.yml config to regulate when Dependabot runs and how many PRs it will open at a time:

https://docs.github.com/en/code-security/reference/supply-ch...

operator-name 3 hours ago||
The refined github extension[0] has some defaults that make the default view a little more tolerable. Past that I can personally recommend Renovate, which supports far more ecosystems and customisation options (like auto merging).

[0]: https://github.com/refined-github/refined-github

12_throw_away 2 hours ago||
I'm a little hung up on this part:

> These PRs were accompanied by a security alert with a nonsensical, made up CVSS v4 score and by a worrying 73% compatibility score, allegedly based on the breakage the update is causing in the ecosystem.

Where did the CVSS score come from exactly? Does dependabot generate CVEs automatically?

indiestack 4 hours ago||
The govulncheck approach (tracing actual code paths to verify vulnerable functions are called) should be the default for every ecosystem, not just Go.

The fundamental problem with Dependabot is that it treats dependency management as a security problem when it's actually a maintenance problem. A vulnerability in a function you never call is not a security issue — it's noise. But Dependabot can't distinguish the two because it operates at the version level, not the call graph level.

For Python projects I've found pip-audit with the --desc flag more useful than Dependabot. It's still version-based, but at least it doesn't create PRs that break your CI at 3am. The real solution is better static analysis that understands reachability, but until that exists for every ecosystem, turning off the noisy tools and doing manual quarterly audits might actually be more secure in practice — because you'll actually read the results instead of auto-merging them.

staticassertion 4 hours ago|
Part of the problem is that customers will scan your code with these tools and they won't accept "we never call that function" as an answer (and maybe that's rational if they can't verify that that's true). This is where actual security starts to really diverge from the practices we've developed in the name of security.
unshavedyak 4 hours ago||
Would be neat if the call graph could be asserted easily.. As you could not only validate what vulnerabilities you are / aren't exposed to, but also choose to blacklist some API calls as a form of mitigation. Ensuring you don't accidentally start using something that's proven unsafe.
viraptor 3 hours ago||
https://bandit.readthedocs.io/en/latest/ can do that for python.
woodruffw 3 hours ago||
I think this is pretty good advice. I find Dependabot useful for managing scheduled dependency bumps (which in turn is useful for sussing out API changes, including unintended semver breakages from upstreams), but Dependabot’s built-in vulnerability scanning is strictly worse than just about every ecosystem’s own built-in solution.
esafak 5 hours ago||
I automate updates with a cooldown, security scanning, and the usual tests. If it passes all that I don't worry about merging it. When something breaks, it is usually because the tests were not good enough, so I fix them. The next step up would be to deploy the update into a canary cluster and observe it for a while. Better that than accrue tech debt. When you update on "your schedule" you still should do all the above, so why not just make it robust enough to automate? Works for me.
FiloSottile 5 hours ago|
For regular updates, because you can minimize but not eliminate risk. As I say in the article that might or might not work for your requirements and practices. For libraries, you also cause compounding churn for your dependents.

For security vulnerabilities, I argue that updating might not be enough! What if your users’ data was compromised? What if your keys should be considered exposed? But the only way to have the bandwidth to do proper triage is by first minimizing false positives.

samhclark 5 hours ago||
This makes sense to me. I guess I'll start hunting for the equivalent of `govulncheck` for Rust/Cargo.

Separately, I love the idea of the `geomys/sandboxed-step` action, but I've got such an aversion to use anyone else's actions, besides the first-party `actions/*` ones. I'll give sandboxed-step a look, sounds like it would be a nice thing to keep in my toolbox.

FiloSottile 5 hours ago||
> I've got such an aversion to use anyone else's actions, besides the first-party `actions/*` ones

Yeah, same. FWIW, geomys/sandboxed-step goes out of its way to use the GitHub Immutable Releases to make the git tag hopefully actually immutable.

conradludgate 5 hours ago|||
https://rustsec.org/
bpavuk 5 hours ago||
> I guess I'll start hunting for the equivalent of `govulncheck` for Rust/Cargo.

how about `cargo-audit`?

mirashii 4 hours ago||
cargo-audit is not quite at an equivalent level yet, it is lacking the specific features discussed in the post that identify the vulnerable parts of the API surface of a library. cargo-audit is like dependabot and others here in that it only tells you that you're using a version that was vulnerable, not that you're using a specific API that was vulnerable.
hobofan 4 hours ago||
Saddly, since it relies on a Cargo.lock to be correct it also is affected by bugs that place dependencies in the Cargo.lock, but are not compiled into the binary. e.g. weak features in Cargo currently cause unused dependencies to show up in the Cargo.lock.
SamuelAdams 5 hours ago|
What’s nice about Dependabot is that it works across multiple languages and platforms. Is there an equivalent to govulncheck for say NPM or Python?
mirashii 4 hours ago||
> Is there an equivalent to govulncheck for say NPM or Python?

There never could be, these languages are simply too dynamic.

woodruffw 3 hours ago|||
In practice this isn’t as big of a hurdle as you might expect: Python is fundamentally dynamic, but most non-obfuscated Python is essentially static in terms of callgraph/reachability. That means that “this specific API is vulnerable” is something you can almost always pinpoint usage for in real Python codebases. The bigger problem is actually encoding vulnerable API information (not just vulnerable package ranges) in a way that’s useful and efficient to query.

(Source: I maintain pip-audit, where this has been a long-standing feature request. We’re still mostly in a place of lacking good metadata from vulnerability feeds to enable it.)

mirashii 1 hour ago||
The thing is that almost always isn't good enough. If it can't prove it, then a human has to be put back in the loop to verify and assert, and on sensitive timelines when you have regulatory requirements on time to acknowledge and resolve CVEs in dependencies.
danudey 3 hours ago||||
With type hints it's possible for code to assert down the possibilities from "who knows what's what" to "assuming these type hints are correct, this function is never called"; not perfect (until we can statically assert that type hints are correct, which maybe we can idk) but still a pretty good step.
robszumski 4 hours ago|||
I commented elsewhere but our team built a custom static analysis engine for JS/TS specifically for the dep update use-case. It was hard, had to do synthetic execution, understands all the crazy remapping and reexporting you can do, etc. Even then it’s hard to penetrate a complex Express app due to how the tree is built up.
tech2 4 hours ago||
For python maybe pip-audit, and perhaps bandit for a little extra?

It doesn't have the code tracing ability that my sibling is referring to, but it's better than nothing.

More comments...