LiteLLM Alternatives: 7 LLM Gateways Compared After the 2026 PyPI Compromise

LiteLLM Alternatives: 7 LLM Gateways Compared After the 2026 PyPI Compromise

LiteLLM is still the default self-hosted LLM router, but the March 2026 PyPI compromise changed how teams evaluate it. We compare Portkey, Bifrost, Kong AI Gateway, Cloudflare AI Gateway, Vercel AI Gateway, OpenRouter and Envoy AI Gateway on hosting model, supply-chain exposure and the governance you actually get.

Disclosure: We run LLM gateways in our own stack and publish the WTF Technology Radar. We have no commercial relationship with any vendor below. Facts last verified 15 November 2026.

Key Takeaways

  • What LiteLLM is — An open-source Python SDK and proxy server from BerriAI that exposes 100+ model providers behind one OpenAI-compatible API, with virtual keys, budgets, fallbacks and spend tracking. It is the most common self-hosted router in 2026.
  • Why teams are re-evaluating it — On March 24, 2026 two PyPI releases (1.82.7 and 1.82.8) shipped a credential stealer that targeted SSH keys, cloud credentials, Kubernetes secrets and .env files. The GitHub source was clean; the payload was injected into the published wheel. A router holds every provider key you own, so the blast radius was large.
  • The alternatives by job — Portkey for a managed control plane (its gateway went fully open source the same week). Bifrost for a Go binary with lower overhead. Kong or Envoy AI Gateway if you already run them for APIs. Cloudflare or Vercel AI Gateway if you deploy there. OpenRouter if you want hosted access with no gateway to run.
  • Bottom line — LiteLLM itself is not the problem; unpinned Python dependencies in the process that holds your keys are. If you stay, pin hashes and run the official container. If you leave, choose by hosting model first, then by governance features.

What LiteLLM actually is

LiteLLM is two things from the same open-source project. The SDK lets Python code call 100+ model providers in the OpenAI request format. The proxy is a server you run between your applications and those providers: it issues virtual keys, enforces per-team budgets, retries and falls back across providers, caches responses and logs spend. For most teams the proxy is the product. It turns "which model, which provider, whose key" into one internal endpoint that every app can use.

That position is also the risk. The proxy holds every provider credential you own, and it runs as a Python service with a large dependency tree. Anything that compromises that process gets the keys.

What changed in March 2026

On 24 March 2026, two LiteLLM releases on PyPI, 1.82.7 and 1.82.8, were found to contain a credential stealer. It went after SSH keys, cloud credentials, Kubernetes secrets, wallets and .env files, encrypted what it found before sending it out, and could start privileged pods across a cluster. The code on GitHub was clean: the payload was added to the published wheels. Researchers linked the attack to the TeamPCP group and an earlier breach of the Trivy CI/CD pipeline, though that link was not confirmed at the time. The bad releases were removed from PyPI.

That reframed the search for LiteLLM alternatives. The question used to be "is there a faster or more polished router". Now it is also "how much of my credential estate do I want sitting in a Python process that installs from a public index".

The 7-gateway shortlist

Self-hosted replacements

  • Portkey: the managed control plane that also self-hosts. On 24 March 2026 Portkey made its whole gateway open source, including governance, observability, auth, cost controls and its MCP gateway, features that used to need the SaaS plan. Best when you want a product, not a project.
  • Bifrost (Maxim AI): an open-source gateway shipped as a Go binary. Maxim publishes benchmarks showing much lower added latency than LiteLLM at high request rates. Treat those as vendor numbers and test on your own traffic. Best for high-throughput, self-hosted routing with a small runtime footprint.
  • Envoy AI Gateway: an open-source project that adds LLM routing, token-based rate limits and provider fallback on top of Envoy. Best if your platform team already runs Envoy or Envoy Gateway.
  • Kong AI Gateway: AI plugins for Kong Gateway: multi-provider routing, semantic caching, prompt guards and token rate limiting. Best if Kong already fronts your APIs and you want LLM traffic under the same policies.

Hosted options

  • Cloudflare AI Gateway: caching, rate limiting, retries, fallbacks and analytics at Cloudflare's edge. Best when your apps already run on Workers.
  • Vercel AI Gateway: one endpoint for many models with fallbacks and spend visibility, wired into the AI SDK. Best for teams deploying on Vercel.
  • OpenRouter: not a gateway you run but a hosted router with the widest model catalog, billed through credits. Best when you want access to many models with no infrastructure at all.

Comparison matrix

LiteLLM in the left column, the seven alternatives summarized on the right. For the ranked head-to-head of the main four, see the best LLM gateway in 2026.

Feature LiteLLMAlternatives shortlist
Hosting model
Self-hostable
Yes. Python proxy, Docker image, Helm chart
Portkey, Bifrost, Kong and Envoy AI Gateway self-host; Cloudflare, Vercel and OpenRouter are hosted only
Managed option
LiteLLM Enterprise and a hosted offering exist; most teams self-host the OSS proxy
Portkey, Kong Konnect, Cloudflare, Vercel and OpenRouter are managed by default
Runtime
Python. Flexible, but pulls a large dependency tree into the process holding provider keys
Bifrost is a single Go binary; Envoy and Kong are compiled proxies; Portkey's gateway is TypeScript
Routing and reliability
Provider coverage
100+ providers in OpenAI format, the widest OSS coverage
OpenRouter has the widest hosted catalog; Portkey fronts 1,600+ models; Kong, Envoy and the platform gateways cover the major providers
Fallbacks, retries, load balancing
Yes, configured per model group in the proxy config
Standard across Portkey, Bifrost, Kong and Envoy; Cloudflare and Vercel offer fallbacks; OpenRouter routes across hosts per model
Gateway overhead
Fine for most apps; becomes visible at thousands of requests per second
Bifrost's own benchmarks claim far lower added latency than LiteLLM (vendor-run, verify on your traffic)
Governance and spend
Virtual keys and budgets
Per-key, per-team and per-user budgets in the OSS proxy
Portkey and Kong strongest; Bifrost has virtual keys and budgets; Cloudflare and Vercel are lighter
SSO, audit logs, RBAC
Mostly behind LiteLLM Enterprise
Portkey moved governance into the open-source gateway in March 2026; Kong gates it behind Enterprise/Konnect
Guardrails and MCP gateway
Guardrail hooks and MCP support exist; depth depends on integrations you wire up
Portkey ships guardrails and an MCP gateway; Kong and Envoy add MCP and AI policies as plugins
Supply-chain exposure
2026 incident history
PyPI 1.82.7 and 1.82.8 backdoored on March 24, 2026; 1.82.6 was the last clean release before the fix
No comparable published incident for the alternatives listed, which proves less than it sounds: check each vendor's release signing
Distribution
pip package plus official Docker image; pinning the image digest avoids the PyPI path
Compiled binaries and containers (Bifrost, Envoy, Kong) or no distribution at all (hosted gateways)
Included Partial Not included Hover for details

If you stay on LiteLLM

  • Do not install from PyPI at deploy time. Run the official container and pin it by digest.
  • If you must use pip, pin hashes (--require-hashes) and mirror packages internally.
  • Give the proxy only the secrets it needs. No shared node credentials, no broad cloud roles.
  • Alert on outbound traffic from the proxy to anything other than your model providers.
  • If you ever ran 1.82.7 or 1.82.8, rotate every credential on that host and check for the published persistence artifacts.

Decision framework

  • Want the widest free provider coverage and can operate Python? Stay on LiteLLM, hardened.
  • Want guardrails, budgets, audit and MCP governance as a product? Portkey.
  • Need a fast self-hosted router with a small runtime? Bifrost.
  • Already run Envoy or Kong for APIs? Envoy AI Gateway or Kong AI Gateway.
  • Deploy on Cloudflare or Vercel? Use their AI Gateway first.
  • Do not want to run anything? OpenRouter.

Frequently asked questions

What is LiteLLM?

LiteLLM is an open-source project from BerriAI with two parts: a Python SDK that calls 100+ model providers in the OpenAI request format, and a proxy server (the LiteLLM gateway) that sits between your apps and those providers. The proxy adds virtual keys, per-team budgets, fallbacks, load balancing, caching and spend logging. It became the default self-hosted LLM router because it is free, covers almost every provider, and gives internal apps one stable API contract.

What happened in the LiteLLM supply chain attack?

On March 24, 2026, Endor Labs and others found that LiteLLM versions 1.82.7 and 1.82.8 on PyPI contained a credential stealer. It harvested SSH keys, cloud credentials, Kubernetes secrets, .env files and similar material, encrypted it and sent it out, and could deploy privileged pods for lateral movement. The source on GitHub was clean; the payload was added to the published packages. Security researchers linked it to the TeamPCP group and a prior compromise of the Trivy CI/CD pipeline, though that link was not confirmed at the time. The malicious releases were pulled from PyPI.

What are the best LiteLLM alternatives?

By job: Portkey if you want a managed control plane with guardrails, budgets and an MCP gateway (its gateway is now fully open source too). Bifrost from Maxim if you want a self-hosted gateway as a single Go binary with lower overhead. Kong AI Gateway or Envoy AI Gateway if your platform team already runs Kong or Envoy for APIs. Cloudflare AI Gateway or Vercel AI Gateway if your apps already deploy there. OpenRouter if you want hosted access to a huge model catalog and do not want to operate a gateway at all.

Is there an open-source alternative to LiteLLM?

Yes. Portkey open-sourced its full gateway, including governance, observability, auth and cost controls, on March 24, 2026. Bifrost is open source and self-hostable. Envoy AI Gateway is an open-source project in the Envoy ecosystem. Kong's AI Gateway plugins run on open-source Kong Gateway, with advanced features in Kong Enterprise. All four can replace the LiteLLM proxy for routing, fallbacks and key management.

LiteLLM vs Portkey: which should I choose?

Choose LiteLLM if you want the widest provider coverage in a free self-hosted proxy and your team is comfortable operating a Python service. Choose Portkey if you want a control plane with prompt management, guardrails, budgets, audit and an MCP gateway without assembling it yourself, either managed or self-hosted. Since March 2026 the open-source gap between them is small; the real difference is whether you want to run a Python proxy or buy a product.

Is it still safe to use LiteLLM?

It can be, with the controls you should apply to any process that holds every provider key you own. Run the official container pinned by digest rather than installing from PyPI at deploy time, pin package hashes if you do use pip, give the proxy only the secrets it needs, and alert on outbound traffic from it. If you installed 1.82.7 or 1.82.8, treat every credential on that host as exposed and rotate it, then look for the published persistence artifacts.

LiteLLM vs OpenRouter?

They solve different problems. LiteLLM is software you run that routes to providers using your own API keys and contracts. OpenRouter is a hosted service you pay that routes to providers using its keys, billed through OpenRouter credits. Pick LiteLLM (or another self-hosted gateway) when you have provider contracts, data residency rules or need internal spend controls. Pick OpenRouter when you want instant access to many models and are fine with a third party in the request path. Our OpenRouter alternatives guide covers that side.

Which LiteLLM alternative is fastest?

Compiled gateways add less latency than a Python proxy. Bifrost publishes benchmarks claiming large overhead reductions versus LiteLLM at high request rates, and Envoy and Kong are built on high-performance proxies. These numbers are vendor-run, and for most applications the model call dominates latency anyway. Gateway overhead matters when you run thousands of requests per second or chain many short calls in an agent loop.

No comments yet. Be the first!

Explore More

Ready to Find the Right AI Tools?

Browse our data-driven rankings to find the best AI tools for your team.