Shai-Hulud Themed Malware Found in the PyTorch Lightning AI Training Library

Posted by j12y 14 hours ago

Shai-Hulud Themed Malware Found in the PyTorch Lightning AI Training Library(semgrep.dev)

356 points | 123 commentspage 2

upupupandaway 12 hours ago|

Not a security guy here. How did the dependency get compromised, exactly? Did they submit a PR into the main repo at github and it was approved by the maintainers? Or just host compromised versions in other mirrors?

andymcsherry 12 hours ago|

Andy from Lightning here. The malicious code was not submitted to the main repo at Github. It appears our PyPi credentials were leaked and compromised packages were published directly there for versions 2.6.2 and 2.6.3

lostmsu 10 hours ago||

I vaguely remember PyPi requiring 2FA about a year and a half ago at least for logins.

If they haven't started yet, they should require 2nd factor for publishing as well.

brahman81 12 hours ago||

Thanks to the community for reporting the security issues with PyTorch Lightning 2.6.2 and 2.6.3 - we're actively looking into it.

In the meantime, please use 2.6.1 until we publish 2.6.4.

For more details: https://github.com/Lightning-AI/pytorch-lightning/security/a...

caycep 12 hours ago||

just to clarify it's not PyTorch, it's the library for this Lightning AI company?

mort96 12 hours ago||

Oh shit I had assumed PyTorch Lightning was affiliated with PyTorch. Not a great name for an unaffiliated third party thing.

lostmsu 12 hours ago|||

Yes

JPKab 9 hours ago||

Yep. Lighting is a blatant, shameless rip off of Jeremy Howard's FastAI library BTW.

notatallshaw 12 hours ago||

> Running pip install lightning is all that is needed to activate

FYI, pip added cooldowns in 26.1:

  * https://discuss.python.org/t/announcement-pip-26-1-release/107108
  * https://ichard26.github.io/blog/2026/04/whats-new-in-pip-26.1/

To use:

  * CLI: pip install --uploaded-prior-to=P1D ...
  * Env Var: PIP_UPLOADED_PRIOR_TO=P1D pip install ...
  * Config: pip config set global.uploaded-prior-to P1D

riteshnoronha16 4 hours ago|

Even if you package manager does not support it, if you generate sboms your implement cooldowns across ecosystems https://www.interlynk.io/resources/cooldowns-with-sboms

bandrami 5 hours ago||

It's crazy to me how just a year or so after xz people were willing to say "sure I'll take this giant black box so unauditable that even it's creators don't really know what's in it and run all my data through it"

0fflineuser 12 hours ago||

The nixpkg from unstable seems to be infected as it s 2.6.2 https://search.nixos.org/packages?channel=unstable&include_h...

minkowski 12 hours ago|

Nixpkgs uses the GitHub source, not the PyPI dist, for lightning; unclear to me from the advisory whether this should also be considered compromised.

andymcsherry 12 hours ago|||

Andy from Lightning here. Thanks for pointing that out, we are updating the CVE. Only the versions from PyPi were affected. The malicious code was not checked into the GitHub repository

deforciant 12 hours ago|||

github is fine, the package was only pushed into pypi directly

ks2048 12 hours ago||

I'm curious what they do with various kinds of credentials if they get access.

I can see trying to steal crypto, but what do they do if they get some AWS credentials? Try to run some crypto mining instances? Try to use your account for other types of crimes? Or is it mainly trying to steal data and then ask for ransoms?

bigfluffydonkey 12 hours ago|

It's always crypto. A client got some AWS credentials stolen and without anyone checking the account, the hacker managed to spin up big EC2 instances across many regions. The bill after a month as I recall was around 100K. Since the activity was clearly fraudulent the bill was forgiven eventually. So remember to lock down your AWS keys permissions...

ajb 7 hours ago|||

When that happened to a former employer AWS was calling us within a day. Worth making sure a real phone number is on there, as that's how they contact you for anything serious (and also if your finance dept decided to change the credit card without telling anyone)

9dev 9 hours ago|||

That; and also, enable the various monitoring and audit features in AWS now; start with CloudTrail. Nothing worse than being affected by this attack, and AWS not having any audit trail available.

throwa356262 13 hours ago||

Advisory, fresh from the owen

https://github.com/Lightning-AI/pytorch-lightning/security/a...

csvance 12 hours ago||

The decision to run all of my experiments in a monorepo with a single uv.lock continues to be validated. I usually only update it a few times a year. It was pinned at 2.6.1 for lightning \o/

cushychicken 8 hours ago|

I was one of probably eight people who played the Emperor: Battle for Dune RTS game, and I always think of the Fremen character sound bite whenever I see the Old Man of the Desert’s true name invoked:

”…for Shai-Hulud!!!”

More comments...