AI Engineering

LiteLLM 1.107.0: a real signature on a backdoored release

Attackers used stolen GitHub CLI tokens to push a backdoored LiteLLM through its signed release pipeline. Here is the exposure check, the rotation list and the gateway fixes.

Glowing vault door drawing in many brass keys while a red thread leaks some away from a cracked box on a conveyor
Glowing vault door drawing in many brass keys while a red thread leaks some away from a cracked box on a conveyor

What happened on 21 October

On Tuesday, LiteLLM's own release pipeline published version 1.107.0 to PyPI and to its container registries, and that version stole credentials. In the project's words, from its official incident report, "LiteLLM's release pipeline was compromised and pushed v1.107.0 to the docker registry and PyPi", and the release "contained compiled malware which steals credentials and loads remote-code execution tools". LiteLLM describes the incident as contained. The investigation was still open at the report's last update, 20:30 UTC on 21 October.

This matters more than the usual typosquat. Nobody had to be tricked into installing a lookalike package. The real package name, the real pipeline and the real signing step all delivered attacker code. If your team runs LiteLLM as its model gateway, that gateway probably holds every provider API key you own. Treat this as a secrets incident first and a dependency incident second.

This is also LiteLLM's second supply-chain compromise this year. The Hacker News reports that the same .pth launch technique appeared in the March 2026 compromise of versions 1.82.7 and 1.82.8.

How signed provenance signed the wrong code

LiteLLM's report sets out the root cause. The attacker held GitHub CLI credentials for three developers. One belonged to an org owner, and the attacker used it to override branch protection and change the private release workflow. A second developer's credential cleared the approval gate. The workflow then built, signed and published artifacts from a lookalike repository, Berri-AI/litellm, with a hyphen. The real org is BerriAI.

StepSecurity's analysis adds a detail worth dwelling on. The v1.107.0 git tag itself is clean. The attacker stacked ten commits on top of it in the lookalike repo, including a data = "cook" line in pyproject.toml. All seven wheels, covering macOS, manylinux, musllinux and Windows, were backdoored.

That is the point the Hacker News thread kept returning to. An attestation proves which workflow published a package. It does not prove which source that workflow built from. Once someone with owner rights can edit the workflow, the signature shows the build came from the official pipeline, which was true. It just says nothing about the code.

How did the tokens leak? This part is LiteLLM's account and has not been independently established. The report says that in specific situations "Claude Code will embed a user's GitHub CLI OAuth token into a response body". According to LiteLLM, those response bodies were logged by an internal LiteLLM deployment that was exposed to the public internet. LiteLLM suspects the initial access came through a recently patched vulnerability but has not confirmed it. Whatever the final finding, the shape of the chain is clear enough to act on: an AI coding agent's credentials ended up in an AI gateway's request logs, and those logs were reachable from outside.

What the payload does

SafeDep's write-up explains why this is worse than a malicious import. The wheels drop a 12-byte cook.pth file containing import cook. Python runs .pth files at interpreter startup, so the stealer runs in every Python process on the host, including ones that never import litellm. A sitecustomize.py forks a detached daemon. Every hour, that daemon reads a domain from a GitHub dead drop (letters9813/pythonclient-test), fetches /codepy from that domain and runs it with exec().

The collection list is broad:

  • SSH, GPG and TLS keys
  • AWS, Azure and gcloud credentials, kubeconfig and Docker config
  • CI secrets for GitLab runners, Jenkins, Vault and Terraform
  • browser passwords
  • the environment of other processes, read from /proc/[pid]/environ
  • tokens for AI coding tools, including Claude Code, Cursor, Copilot, Gemini CLI and Codex
  • dumps of the proxy's LiteLLM_CredentialsTable, which holds provider API keys

The data is encrypted and POSTed to www.litellm[.]dev/healthcheck.

Attackers have already used what they took. Wiz observed sts:GetCallerIdentity calls followed by Bedrock model listing, which it calls "consistent with LLMjacking". In some environments it also saw IAM and CloudTrail enumeration.

A five-minute exposure check

LiteLLM's first instruction is the counter-intuitive one: do not start with pip install. Starting Python re-triggers the malware, and so does any tool that runs on Python, pip included. Do the first pass from the shell.

Start with the files the wheels drop into site-packages:

sudo find / -xdev \( -name 'cook.pth' -o -name 'cook.py' -o -name 'sitecustomize.py' \) 2>/dev/null

Then look for the host indicators from Wiz and LiteLLM: the marker file, the daemon's lock file, and any process carrying the stealer's environment variable:

ls -la /tmp/.pc_once_72129118c59f5636c0bb /tmp/remote_code_loader.lock 2>/dev/null
sudo grep -l 'PRIVCOLLECT_ACTIVE=1' /proc/*/environ 2>/dev/null

A sitecustomize.py can be legitimate on some systems, so open any hit before you delete it. Any cook.pth is not legitimate.

Next, check containers. The affected tags were v1.107.0, latest, main-latest and main-stable on the litellm, litellm-database and litellm-non_root images. If you pulled any of those floating tags on 21 October, compare the digest you are running with the one you ran before:

docker image ls --digests | grep -i litellm

Then check the time window. Per LiteLLM's timeline (all UTC, 21 October), branch protection was overridden at 10:08. Seven malicious wheels went to PyPI at 11:39–11:40, and container images followed at 11:46. PyPI had removed the release by 13:23. GHCR tags were confirmed unavailable at 13:51, and Docker Hub tags were removed or repointed by 14:20. Any unpinned install, CI build or image pull between 11:39 and 14:20 UTC could have picked up the bad version. Search your CI logs and lockfile diffs for 1.107.0 across that window.

If you find it, LiteLLM's rule is blunt: "If 1.107.0 was ever installed on a host with outbound internet, assume the exfil succeeded."

Clean up and rotate

From the shell, LiteLLM's remediation is:

  1. Delete cook.pth, cook.py and sitecustomize.py, along with their .pyc files.
  2. Kill the daemon holding /tmp/remote_code_loader.lock.
  3. Rebuild pinned to 1.106.2.
  4. Rotate the IAM role, every provider key in the proxy database, the master and virtual keys, and the DATABASE_URL password.
  5. Review CloudTrail for GetCallerIdentity.

The rotation list is where the gateway's role shows. A LiteLLM proxy is not one secret. It is a keyring for every model provider account you use, plus the virtual keys you issued to your own teams. The stealer dumped that table directly. Rotate every row, not only the keys you remember adding.

Go beyond LiteLLM's own list for anything else on the host. The stealer also read cloud credentials, kubeconfig, CI secrets, SSH keys and coding-agent tokens. On a developer laptop or a shared build runner, that is most of what the machine could reach.

LiteLLM's side has moved too. Director of security Oliver Jensen told The Hacker News that "All tokens which were involved in this attack have been revoked." As a stopgap, the company removed every member of its GitHub org except one unaffected user.

Treat the gateway like a vault

The structural lesson is about where secrets pile up. An LLM gateway centralises every model credential a team owns, and by design it sees every request and response. That makes it a secrets vault whether you planned it that way or not. Here is what I would change this week:

  • Pin by version and by digest. Pin Python dependencies with hashes and container images by @sha256: digest, not by latest or main-stable. A floating tag is a standing invitation to whatever the pipeline published most recently. The HN debate over 24–48 hour cooldowns is worth having. In this case PyPI had pulled the release in under two hours, but a cooldown only helps if something actually notices the bad release inside the window.
  • Scrub gateway logs. If your gateway logs request and response bodies, redact credentials before they hit disk, or turn body logging off. By LiteLLM's account, this whole chain started with tokens sitting in logged response bodies.
  • Keep internal gateways internal. An admin UI or log store on the public internet is an attack surface holding every key you have. Put it behind a VPN or an identity-aware proxy.
  • Scope what agents can see. Ask the question the HN thread asked: why did a coding agent have a GitHub token in its context at all? Give agents fine-grained tokens with the minimum scope, and do not let an org owner's gh session be the one an agent works inside.
  • Watch the release workflow, not just the code. Here, branch protection and the approval gate both fell to stolen credentials. Alert on any change to branch protection rules or release workflows, so an override like the one at 10:08 is noisy instead of quiet.

Signing still has value. It just answers a narrower question than most of us assumed. The rest of the answer comes from treating the model gateway as the most sensitive service in the stack, because for a lot of teams it now is.

Related reading

Comments (0)

No comments yet. Start the conversation.

Never shown, never emailed — stored only as a hash.