Top
Best
New

Posted by dot_treo 16 hours ago

Tell HN: Litellm 1.82.7 and 1.82.8 on PyPI are compromised(github.com)
About an hour ago new versions have been deployed to PyPI.

I was just setting up a new project, and things behaved weirdly. My laptop ran out of RAM, it looked like a forkbomb was running.

I've investigated, and found that a base64 encoded blob has been added to proxy_server.py.

It writes and decodes another file which it then runs.

I'm in the process of reporting this upstream, but wanted to give everyone here a headsup.

It is also reported in this issue: https://github.com/BerriAI/litellm/issues/24512

531 points | 385 commentspage 3
abhisek 13 hours ago|
We just analysed the payload. Technical details here: https://safedep.io/malicious-litellm-1-82-8-analysis/

We are looking at similar attack vectors (pth injection), signatures etc. in other PyPI packages that we know of.

sschueller 15 hours ago||
Does anyone know a good alternate project that works similarly (share multipple LLMs across a set of users)? LiteLLM has been getting worse and trying to get me to upgrade to a paid version. I also had issues with creating tokens for other users etc.
howardjohn 2 hours ago||
agentgateway.dev is one I have been working on that is worth a look if you are using the proxy side of LiteLLM. It's open source part of the Linux foundation.
treefarmer 11 hours ago|||
If you're talking about their proxy offering, I had this exact same issue and switched to Portkey. I just use their free plan and don't care about the logs (I log separately on my own). It's way faster (probably cause their code isn't garbage like the LiteLLM code - they had a 5K+ line Python file with all their important code in it the last time I checked).
sschueller 14 hours ago|||
I just found https://github.com/jasmedia/InferXgate which looks interesting although quite new and not supporting so many providers.
redrove 14 hours ago|||
Bifrost is the only real alternative I'm aware of https://github.com/maximhq/bifrost
sschueller 14 hours ago|||
Virtual Keys is an Enterprise feature. I am not going to pay for something like this in order to provide my family access to all my models. I can do without cost control (although it would be nice) but I need for users to be able to generate a key and us this key to access all the models I provide.
NeutralCrane 6 hours ago||
I don’t believe it is an enterprise feature. I did some testing on Bifrost just last month on a free open source instance and was able to set up virtual keys.
beanaroo 11 hours ago|||
We have tried reaching out to their sales multiple times but never get a response.
river_otter 15 hours ago|||
github.com/mozilla-ai/any-llm :)
tacoooooooo 15 hours ago|||
pydantic-ai
thibault000 13 hours ago||
[dead]
Shank 13 hours ago||
I wonder at what point ecosystems just force a credential rotation. Trivy and now LiteLLM have probably cleaned out a sizable number of credentials, and now it's up to each person and/or team to rotate. TeamPCP is sitting on a treasure trove of credentials and based on this, they're probably carefully mapping out what they can exploit and building payloads for each one.

It would be interesting if Python, NPM, Rubygems, etc all just decided to initiate an ecosystem-wide credential reset. On one hand, it would be highly disruptive. On the other hand, it would probably stop the damage from spreading.

post-it 9 hours ago|
It'll only be disruptive to people who are improperly managing their credentials. Cattle not pets applies to credentials too.
agentictrustkit 3 hours ago||
I think this gets a lot worse when we look at it from an agentic perspective. Like when a dev person hits a compromising package, there's usually a "hold on, that's weird" moment before a catastrophe. An agent doesn't have that instinct.

Oh boy supply chain integrity will be an agent governenace problem, not just a devops one. If you send out an agent that can autonomously pull packages, do code, or access creds, then the blast radius of compromises widens. That's why I think there's an argument for least-privilege by default--agents should have scoped, auditable authority over what they can install and execute, and approval for anything outside the boundaries.

cpburns2009 15 hours ago||
You can see it for yourself here:

https://inspector.pypi.io/project/litellm/1.82.8/packages/fd...

jbkkd 14 hours ago||
Two URLs found in the exploit: https://checkmarx.zone/raw https://models.litellm.cloud/
santiago-pl 9 hours ago||
It looks like Trivy was compromised at least five days ago. https://www.wiz.io/blog/trivy-compromised-teampcp-supply-cha...
kevml 15 hours ago||
More details here: https://futuresearch.ai/blog/litellm-pypi-supply-chain-attac...
ddp26 7 hours ago||
Yeah, this was my team at FutureSearch that had the lucky experience of being first to hit this, before the malware was disclosed.

One thing not in that writeup is that very little action was needed for my engineer to get pwnd. uvx automatically pulled latest litellm (version unpinned) and built the environment. Then Cursor started up the local MCP server automatically on load.

r2vcap 6 hours ago||
Does the Python ecosystem have anything like pnpm’s minimumReleaseAge setting? Maybe I’m being overly paranoid, but it feels like every internet-facing ecosystem should have something like this.
mark_l_watson 14 hours ago||
A question from a non-python-security-expert: is committing uv.lock files for specific versions, and only infrequently updating versions a reasonable practice?
Imustaskforhelp 14 hours ago|
(I am not a security expert either)

But, one of the arguments that I saw online from this was that when a security researcher finds a bug and reports it to the OSS project/Company they then fix the code silently and include it within the new version and after some time, they make the information public

So if you run infrequently updated versions, then you run a risk of allowing hackers access as well.

(An good example I can think of is OpenCode which had an issue which could allow RCE and the security researcher team asked Opencode secretly but no response came so after sometime of no response, they released the knowledge in public and Opencode quickly made a patch to fix that issue but if you were running the older code, you would've been vulnerable to RCE)

mark_l_watson 12 hours ago||
Good points. Perhaps there is a way to configure uv to only use a new version if it is 24 hours old?
arwt 5 hours ago||
You can. See: https://docs.astral.sh/uv/reference/cli/#uv-run--exclude-new...

How you use it depends on your workflow. An entry like this in your pyproject.toml could suffice:

  [tool.uv]
  exclude-newer = "5 days"
postalcoder 15 hours ago|
This is a brutal one. A ton of people use litellm as their gateway.
eoskx 14 hours ago||
Not just as a gateway in a lot cases, but CrewAI and DSPy use it directly. DSPy uses it as its only way to call upstream LLM providers and CrewAI falls back to it if the OpenAI, Anthropic, etc. SDKs aren't available.
Imustaskforhelp 15 hours ago||
Do you feel as if people will update litellm without looking at this discussion/maybe having it be automatic which would then lead to loss of crypto wallets/ especially AI Api keys?

Now I am not worried about the Ai Api keys having much damage but I am thinking of one step further and I am not sure how many of these corporations follow privacy policy and so perhaps someone more experienced can tell me but wouldn't these applications keep logs for legal purposes and those logs can contain sensitive information, both of businesses but also, private individuals perhaps too?

daveguy 15 hours ago||
Maybe then people will start to realize crypto isn't even worth the stored bits.

Irrevocable transfers... What could go wrong?

More comments...