LiteLLM

LiteLLM, made by BerriAI, is an open-source Python library and self-hosted proxy, also called an LLM gateway. It puts more than 100 large language model providers behind one OpenAI-compatible interface. These include OpenAI, Anthropic, Vertex AI and Bedrock. Teams run the proxy in their own infrastructure, usually from the official Docker image. Client apps then call one endpoint, and the proxy handles routing, load balancing and fallbacks between models. The gateway also provides virtual API keys, budgets, rate limits, and spend tracking by key, user and team. It adds caching, guardrails such as PII masking, an admin UI, and MCP and A2A agent gateway support. It sends logs to observability tools including Langfuse, MLflow, Helicone, Lunary and OpenTelemetry. The open-source edition costs nothing to license, but you pay for the servers it runs on and for the people who run them. Virtual keys, spend tracking and the admin UI need a database, and production setups typically add Redis. The Enterprise tier is priced on request and comes with a 30-day trial key. It adds SSO and SCIM, RBAC, audit logs, secret-manager key rotation, air-gapped and multi-region deployment, and 24/7 support with response targets. It suits engineering teams that want to control model access and costs inside their own environment. The main limitation is operational: you run, secure and upgrade it yourself. Independent reviews report added latency and regressions at high concurrency. In October 2026 the company disclosed a supply-chain compromise of package version 1.107.0.