Posted by kurmiashish 4 hours ago
There's plenty of comments mentioning delay lines, and the other good stuff pnpm (and others) have implemented in response to protect package consumers.
That bit that's getting less conversation is the tools on the package maintainer side:
- MFA for publishing: https://docs.npmjs.com/requiring-2fa-for-package-publishing-...
- trusted publishers, available for about a year: https://docs.npmjs.com/trusted-publishers
And as of recently, staged publishing, essentially combining the best of both those features: https://docs.npmjs.com/staged-publishing
Now you can: - Publish from CI, without static credentials
- AND require a maintainer to approve it using MFA before it actually goes live to the registry
If you want you can still use something like GitHub Actions Environments protection to require multiple approvals, or a time delay, on the CI side.
We need to encourage the community to adopt these publishing protections or this will continue to be an issue.
The bug here isn't that we need to better authenticate already-trusted upstreams for packages, it's that the upstreams cannot be trusted as the sole source for security at all. Upstreams are a bunch of hackers[1] who aren't really interested in, nor will ever be good at, solid release engineering practices.
But some people are! The solution in the Linux world (and the one that saved us from xz-utils) is that there is a second level of human beings responsible for reviewing, auditing, packaging, and customizing those hacker-generated upstreams for the benefit of their users. These people have different eyes, different consumer requirements and different quality metrics. And they catch bugs and malfesance that the upstreams aren't prepared to do.
NPM (and cargo/PyPI et. al.) continues to think it can short circuit this requirement for human labor. It can't.
[1] In NPM's particular ecosystem, a bunch of web jockeys used to extremely fast release processes, loose compatibility requirements, and extreme reliance on reuse. This really explains why we see this with node packages more than Python or Rust: older and more conservative programmers just don't have as many rakes to step on.
I know people have opinions about cooldowns, but they would have saved you from axios, tanstack, (+ @redhat-cloud-services) and many other recent npm supply chain attacks. If you have Artifactory / Nexus, you probably already have cooldowns, but it's easy to set up if you don't. Why cooldowns? Most npm (or pypi) compromises were taken down within hours, cooldowns simply mean - ignore any package with release date younger than N days (1 day can work, 3 days is ok, 7 days is a bit of an overkill but works too)
How to set them up?
- use latest pnpm, they added 1 day cooldown by default https://pnpm.io/supply-chain-security
- or if you want a one click fix, use https://depsguard.com (cli that adds cooldowns + other recommended settings to npm, pnpm, yarn, bun, uv, dependabot and, disclaimer: I’m the maintainer)
- or use https://cooldowns.dev which is more focused on, well, cooldowns, with also a script to help set it up locally
All are open source / free.
If you know how to edit your ~/.npmrc etc, you don't really need any of them, but if you have a loved one who just needs a one click fix, these can likely save them from the next attack.
Caveat - if you need to patch a new critical CVE, you need to bypass the cooldown, but each of them have a way to do so (described in detail in depsguard.com / cooldowns.dev) In the past few months, while I don't have hard numbers, it seems more risk has come from Software Supply Chain attacks (malicious versions pushed) than from new zero day CVEs (even in the age of Mythos driven vulnerability discovery)
> Exact version pinning — specifying precise versions (1.0.0, ==1.0.0, =1.0.0, = 5.31.0) rather than ranges (^, ~>, >=) in package manifests. Ranges allow any version satisfying the constraint to be resolved at install time; exact pins mean only one version is ever valid.
My understanding is that pinning the dependency within the manifest isn't the mechanism that prevents the version from changing across installs -- it's the lockfile that accomplishes this.
This feels like a very very small group of people; and people who really could do with opening the file and adding the line.
So yes, everyone could open a file and edit it, also everyone could watch a youtube video on how to do X and yet choose to have someone else do it for them :)
It is. Changing oil requires a place where you have sufficient access to the vehicle to drain it; the right equipment; the right disposal solutions. Most people who have cars do not have that. And it takes significantly more time to change your own oil than to have someone else do it as part of other specialist maintenance.
> Think of QR codes, people hardly used them for many years, because you needed to download an app for it, small step. It only started to catch up when you had it built in the camera app in most providers.
Exactly. Using a QR code app required specific knowledge of the app, an internet connection, some time, knowledge of how and when to use it, and something to use it with - the barrier of which surpassed the convenience gained from the QR code.
> So yes, everyone could open a file and edit it, also everyone could watch a youtube video on how to do X and yet choose to have someone else do it for them :)
I'm struggling to find a non-contrived group of people who:
- do not know how to open and edit a file on their system
- do use npm
- would find installing pnpm or running `sudo install -d -m 0755 /etc/apt/keyrings; curl -fsSL https://depsguard.com/apt/gpg.key | sudo gpg --dearmor -o /etc/apt/keyrings/depsguard.gpg; echo "deb [signed-by=/etc/apt/keyrings/depsguard.gpg] https://depsguard.com/apt stable main" | sudo tee /etc/apt/sources.list.d/depsguard.list >/dev/null; sudo apt update; sudo apt install depsguard` simpler
Of course, cooldowns.dev is a very long winded way of telling someone to run `npm config set min-release-age=3`, which is the simplest.
> a place where you have sufficient access to the vehicle to drain it
Probably the only valid argument for people who park on the street.
> the right equipment
One $5 wrench, one $10 filter wrench (optional). One set of ramps ($40), or jack stands ($30) if you already have a jack. One drain pan, $10 (or free if you're resourceful). Total cost max $65. Cheaper if you look for deals, buy used, borrow from a friend. If you can't afford $65 once to save money in the long run while owning a car, you probably should've bought a cheaper car.
> the right disposal solutions
Every oil change requires a jug of oil to be purchased. You can drain your used oil into this jug and then dispose of it along with your other household hazardous waste. This is not hard.
> Most people who have cars do not have that.
I might believe this for a place to do an oil change, maybe. I struggle to believe most, but I would believe many. Aside from that, if you don't have those things, you are choosing not to have them.
Which is kind of the point. None of these things are hard, at all. The majority of car owners 100 years ago could adjust their own timing, clean distributor points, replace belts, etc. because if they couldn't, they'd be calling for a tow truck every few hundred miles. Those are all harder, and things have only gotten easier with time. If you can't do them, you are choosing not to, because there's an even easier solution - spending more money and getting someone else to do it for you.
The issue is that Claude Code also will be super happy to npm install axios / tanstack etc unless you explicitly tell it to add cooldowns.
The JS ecosystem is really, really complicated, so any non-trivial app is going to use multiple bundlers, node runtimes, native runtimes, etc, etc, etc.
Every one of those has a different opinion about how to spell "cooldown".
On top of that, there's the bootstrapping issue of "I want to install the N pieces of ecosystem sprawl that read the .[p]npmrc that have the cooldown directive in them. How do I do that with a cooldown?" (Where N is unknowable, because of course it is.)
by now, you should have received the feedback about why cooldowns don't make sense and why nobody is adopting them. look, you are writing an expression of the reason why right there.
- Most companies I know have a 24 hours (at least) cooldown via their Artifactory / Nexus. They have ways to bypass it for urgent CVEs
- pnpm just adopted 24 hours cooldown as default, based on community feedback.
- checking every update of every dependency to see if is a relevant urgent security update
- checking every update of every dependency to see if it turns out to be a supply chain exploit
am i still checking every update of every dependency? there's no heuristic here. either you check them all, or you get randomly exploited - either by using known vulnerable software or from supply chain attacked software.
https://gist.github.com/mcollina/b294a6c39ee700d24073c0e5a4e...
The package axios was compromised, and hijacked the author's credentials, so every attempt at a fix was unfixed. https://www.trendmicro.com/en_us/research/26/c/axios-npm-pac...
The xz utility was backdoored for 2 months: https://gigazine.net/gsc_news/en/20240403-timeline-of-xz-ope...
A student researcher took over Python ctx and PHPass package maintainership, pushing out malicious changes, and that took over 7 days to be detected and fixed: https://infosecwriteups.com/how-i-hacked-ctx-and-phpass-modu...
Kaspersky found multiple PyPI packages that had been exploited for more than a year: https://www.kaspersky.com/about/press-releases/kaspersky-unc...
"LoftyLife" packages were exploited for several months: https://securelist.com/lofylife-malicious-npm-packages/10701...
Now that the attack window has changed to 7 days, all new exploits like these will come with time bombs to not trigger until 8 days.
Many automated scanners use static code analysis rather than run the installation script. Not all of them are caught, but a good part of them are and you'd be saved by a delay.
Many places run analyzers on published code; many security users have reason to shorten the period. The default period becomes the period where white hats have a chance to detect it and stop it passing the threshold.
`pip install --uploaded-prior-to P7D pre-commit`
https://pip.pypa.io/en/stable/cli/pip_install/#cmdoption-upl...
https://github.com/NuGet/Home/issues/14657#issuecomment-3573...
huh? what do you suggest instead?
To give you a context, I get 20-30 PRs a week across all my repos with potentially hundreds of packages (non distinct) from dependabot. I give it a cursory look and try to get a summary of changes. Do I evaluate every single package update? Nope.
There is something to be said that Microsoft should be scanning packages pre-release. They aren't, though, so for right now there is a ton of value with very little downside if people implement a one week cooldown period.
To answer your question directly, though. If everyone else moves to a one week cooldown, I would absolutely suggest a two week cooldown is a good idea. Being the "slow" moving organization is a good security trade-off so long as you don't take it to extremes and have escape hatches when you actually need to be moving quickly.
Of course, this also assumes that Microsoft's internal scanners are much better than the scanners available to the attackers, since any reasonable attacker is going to just run their obfuscated code through a scanner as part of their CI job. (And maybe even use the MS scanner as an oracle by submitting fragments to NPM to see which pieces of their exploit chain get flagged.)
Waiting until everyone else canaries is much stronger, but even that doesn't work on a targeted attack.
* The JS ecosystem has been and will most likely continue to be fast-moving, so it's quite a safe assumption that at no point will a quarantine period be wide-spread.
* This quarantine period is for (semi-)automated scanners to catch the issue. Although considering the above there will always be a non-zero amount of end-user canaries as well.
* Maybe NPM should run scanners before distributing malware?
* If the ecosystem by any chance adopts a week-long quarantine period, you'd be safer if you applied a longer quarantine period.
I suspect there's always a human checking these results. If NPM straight out rejects an update due to suspected malware, they might end up rejecting correct updates as well. If they grant some "safe" patterns a special pass, they might get exploited.
So I think this only works if you have security scanners that are well-maintained and kept in secret. NPM folks could of course co-operate with some security companies to have a first stab with the releases before they are put to public access. At some point some parties might start want to have monetary compensation for such an arragnement, though.
https://github.com/rust-lang/rfcs/blob/master/text/3923-carg...
For example, is a crypto miner actually an attack? If the package presents itself as a miner, then no. Is connections to other repositories an attack? Again, depends on what the package does. Connections to some other hostname? Depends.
There is still a lot of human analysis that occurs in making the call that an attack is occurring.
None of this is to say I think Microsoft shouldn't be doing something as part of the release process on NPM. However, there is real value in giving more independent third parties a window to do things semi-manually.
I think these attackers are unlikely to add a delay in the first place because the chance of their attack being found out before it activates would be too high. They seem to generally work on the assumption that they have a day or so before the package is yanked (e.g: from maintainer noticing their account is compromised) so need to move fast.
1. Dependency cooldowns of 1-2 days seem to be extremely effective without negatively impacting your ability to patch for CVEs.
2. Anywhere you have `npm install` or `npm test` or anything where code executes, that should happen in an environment that has no privileges. In your github actions you can do this semi-straightforwardly by using two separate jobs - one to build the artifacts and test them, another to do any sort of publishing, signing, etc. If you use AI, add a skill / guidance to enforce this pattern.
3. If you use Github Actions, install the latest version of zizmor. It will significantly improve your posture.
(2) means that you are no longer "wormable", which is a massive part of the problem that we have today. (1) gives companies more time to respond to the attacks.
There are some vendors in this space that you can and should evaluate as well.
I’m running a maven proxy locally. All builds happen inside containers. I only use public repos for python, npm, and go. So these builds happen also in containers but don’t need a repository proxy.
I'd suggest both. Cooldown for 1-2 days is very cheap and you likely won't even notice it, so it's quite harmless and from what I've seen even just 24 hours is enough to let security companies pick up malware.
But yeah, isolation is a must-have.
Or as us or companies to wrap the build tools to provide the wrapping for them.
What if it gets compromised?
More of a joke. But was funny after saying that new packages should be delayed.
ALL the agentic orchestrators like codex, claude-code, etc. seem to do this by default.
It does make sense that the right way would be to fork every dependency you use and install from your own repo reviewing and merging from upstream as needed. Would be a giant PITA though. :)
I even managed to make that part of the workflow on one team I worked with but several other teams since thought it was a crazy idea. :)
Node.js doesn't have good support for regex, handling files, streams, serving static html, routing, operations on lists/dicts.
That is, at least what we do, in theory. In practice, we cross fingers and let the LLM pick dependencies, are satisfied if it just works and we either update our deps frequently or infrequently.
https://news.ycombinator.com/item?id=47017833
Well, now with an irony, but sadly, of course.
I have never seen a project that uses npm and has only dozens of dependencies. Normal numbers are in the 10s of thousands (including different versions of some deps).
I'm trying to do all work in dev containers (or other sandboxes), limiting the blast radius if I'm unlucky enough to be hit by an exploit. The attackers may get a Claude token, but they won't easily be able to escape the container and scan my home dir.
Cooldowns and allow-listing of installer scripts are good additions to layered security, especially for CI. However, I think the fundamental thing that needs to change is the OS permissions model. The default of trusting third-party software with everything your user has access is no longer workable.
https://www.stepsecurity.io/blog/multiple-redhat-cloud-servi...
Thankfully, it's on by default since v11.
min-release-age=5