LiteLLM
Self-hosted AI gateway for 140+ LLM providers, MCP servers, and agents — one OpenAI API with cost control.
LiteLLM is the best pick for platform teams that want a self-hosted AI gateway with deep provider variety and granular spend controls. The Rust rewrite and Auto Router v2 give it a real performance edge at scale. Skip it if you need a managed, no-ops cloud service or built-in agent workflows — Portkey or Helicone are better fits there.
Verified 9d ago · liveness 78/100 · cite: rightaichoice.com/tools/litellm
- Platform teams providing unified LLM access to developers
- Organizations needing cost tracking and chargebacks per team/org
- Multi-provider LLM deployments requiring fallback and load balancing
- Enterprises wanting an OpenAI-compatible drop-in replacement for custom models
- Single-provider shops that don't need multi-model abstraction
- Projects requiring advanced prompt chaining or agent workflows (gateway only)
- Organizations needing a managed, no-ops cloud service
We scan live Reddit threads, YouTube comments, X posts, G2 reviews and other communities — and hand you an honest verdict in under a minute.
- Honest verdict, not marketing
- Real pros & cons from real users
- Attributed quotes with receipts
3 free scans · no card needed
Skip LiteLLM if you need a managed, no-ops cloud service, built-in agent workflows, or if you're a single-provider shop that doesn't need multi-model abstraction.
Self-hosting costs (infrastructure, monitoring, patching) are on you; the software is free but you pay for the ops.
LiteLLM's free self-hosted tier is a no-brainer for any team that can manage its own infrastructure. Enterprise is priced by annual request capacity, which can be more cost-effective than per-token pricing from managed gateways like Portkey or Helicone, but only if you have the DevOps resources to run it.
In short
LiteLLM — Self-hosted AI gateway for 140+ LLM providers, MCP servers, and agents — one OpenAI API with cost control. Best for Platform teams providing unified LLM access to developers, Organizations needing cost tracking and chargebacks per team/org, Multi-provider LLM deployments requiring fallback and load balancing. Free to use.
What's new in LiteLLM
Checked 9 days agoAcross the latest 3 updates: 3 feature updates.
Shadow Evaluations: Test the Auto-Router on Your Own Production Traffic
Shadow evaluations duplicate a sampled slice of production traffic through an auto-router; router matched or beat existing model on 88.1% of judged responses.
Day 0 support: Gemini 3.7 Flash
LiteLLM adds day-0 support for Gemini 3.7 Flash, tracking launch pricing across Google AI Studio and Vertex AI.
51% Cost Savings Reported From a Live Production Deployment
Customer reports $12,249 saved over 272,876 requests and 7.08B tokens using Auto Router in production with 450+ users.
Viability Score
How well maintained and how widely used is LiteLLM? Built from what the vendor actually publishes (docs, changelog, tutorials, integrations, pricing), whether the site is live, and how much real users discuss it. How we calculate this
Last calculated: September 2026
How we score →Key Features
- One OpenAI-compatible API to 140+ providers and 1,800+ models
- Day-0 support for new models (e.g., Gemini 3.7 Flash, Claude Opus 5)
- Rust-based gateway core for low latency and memory overhead
- Auto Router v2 with complexity, semantic, and adaptive routing
- Router Plugins for custom routing signals
- Virtual keys, teams, and scoped access with SSO
- Spend tracking by key, user, team, and org
- Budget caps and rate limits (RPM/TPM)
- LLM fallbacks across providers with cooldowns
- Load balancing and retries across deployments
- MCP server and agent support through the same gateway
- Guardrails integrations (Presidio, Lakera, Aporia, Bedrock Guardrails)
- Observability via Prometheus, Langfuse, OpenTelemetry
- Self-hosted and air-gapped deployment
- Shadow evaluations of Auto-Router on production traffic
About LiteLLM
LiteLLM is an open-source AI gateway that gives platform teams a single, self-hosted entry point to their entire AI stack: LLMs, MCP servers, and agents. It sits between your internal apps and hundreds of model providers, exposing one OpenAI-compatible API to 140+ providers and 1,800+ models. That means developers across the org can build without waiting on procurement — virtual keys and SSO handle access, while the gateway enforces team-based budgets, rate limits, and automatic fallbacks. If your stack spans OpenAI, Anthropic, Vertex, or Bedrock, swapping models is a config change, not a code rewrite. The gateway is built for production scale. Its core is being rewritten in Rust to cut latency and memory overhead, and recent benchmarks show the lowest overhead among comparable gateways. Auto Router v2 combines complexity, semantic, and adaptive routing into a single router — it uses an LLM classifier and Thompson sampling to send each request to the right model, and you can extend it with Router Plugins for custom routing signals. Day-0 support means new models, like Claude Opus 5, are available the day they ship, which is why teams at Netflix, NVIDIA, Lemonade, and Okta run it in production. Cost control is a first-class feature. You can track spend by key, user, team, and org, set budget caps and per-tag budgets, and enforce rate limits (RPM/TPM). The platform routes over a billion requests and surfaces everything in Prometheus, Langfuse, and OpenTelemetry, so observability fits your existing stack. Guardrails integrations with Presidio, Lakera, Aporia, and Bedrock Guardrails add a layer of safety. LiteLLM is free to self-host forever. The Open Source tier gives you virtual keys, teams, spend tracking, budgets, rate limits, fallbacks, request logging, and Prometheus metrics. For governed production use, Enterprise adds SSO/SCIM, audit logs, secret managers, a multi-region control plane, and 24/7 support, priced by annual request capacity — never per token.
Behind the Verdict
LiteLLM is the de facto standard for self-hosted AI gateways, and for good reason: it gives platform teams one OpenAI-compatible API to 140+ providers and 1,800+ models. The modular design means you can start with a single provider and expand without rewriting app code. The virtual keys and team-based budgets solve a real pain for orgs that need to control spend and provide chargebacks. Strengths: The breadth of provider support is unmatched, with day-0 support for new models — recent examples include Gemini 3.7 Flash and Claude Opus 5. The Rust rewrite and Auto Router v2 deliver real performance gains; benchmarks show the lowest overhead among comparable gateways and routing that matches Opus-level quality at up to 27% lower cost. The open-source core is free to self-host, which attracts a strong community and transparent code. Weaknesses: It's not a managed service — you're responsible for hosting, uptime, and security. Configuration can become complex in large orgs with many routing rules. The July 2026 supply chain attack that hit 2,488 organizations underscores the risk of self-hosting; you must stay current with patches. Enterprise pricing is not public, and advanced support requires a sales conversation. Where it fits: platform teams at mid-to-large companies that need unified LLM access, cost tracking, and fallback resilience. Where it doesn't: small teams that just need a quick API key for a single provider, or orgs that want built-in agent orchestration — for that, consider Portkey or Helicone.
Researching LiteLLM? Get your full AI stack in 60 seconds.
Free, no signup — tell us your goal and get tools matched to your budget & existing stack.
Real-world workflow fit
Concrete scenarios for the personas LiteLLM actually fits — and what changes day-one when you adopt it.
Set up LiteLLM proxy in a Docker container, configure virtual keys for each team, and route traffic to OpenAI and Anthropic. Within hours, developers get a single API endpoint and spend dashboards.
Outcome: Teams can ship AI features without waiting for API keys; the platform team gets visibility into per-team spend and can set budgets to prevent cost overruns.
Enable Auto Router v2 with preset configurations for OpenAI and Anthropic. Use shadow evaluations to compare routing decisions against existing models on production traffic.
Outcome: Auto Router matches Opus-level quality on many tasks while reducing cost by up to 51%, as reported in production deployments.
Deploy LiteLLM air-gapped, integrate with SSO/SCIM and a secrets manager, and set up audit logs for compliance. Route all LLM traffic through the gateway to enforce guardrails.
Outcome: The org gets centralized governance without sending data to external parties, and can enforce PII masking and prompt-injection prevention.
Use Cases
- Centralize LLM access with virtual API keys for every team in the organization.
- Add automatic fallback from OpenAI to Azure OpenAI during outages with zero code changes.
- Track per-team LLM cost and enforce monthly budgets without writing billing code.
- Run a local Ollama model behind an OpenAI-compatible endpoint for rapid prototyping.
- Migrate existing projects to the proxy using pass-through endpoints without translation.
- Route requests to the most cost-effective model with Auto Router, cutting costs by up to 51%.
Models Under the Hood
as of 2026-08-30
Limitations
- The gateway adds a network hop, which may introduce latency, though the Rust core aims to reduce overhead.
- Configuration can become complex for large organizations with many routing rules.
- Enterprise pricing is not public and requires a sales call.
- Security maintenance is critical; an August 2026 supply chain attack affected 2,488 organizations, and a July 2026 update included 38 security fixes.
as of 2026-08-28
Verification history
We have re-verified LiteLLM 17 times since . Each pass re-reads the vendor's own pages and re-checks every listed field against that evidence; passes where nothing had changed are marked as such.
- — re-verified summary, description, our verdict, our analysis, pricing model, pricing tiers, features, integrations, who it suits, who should skip it
- — re-verified summary, description, our verdict, our analysis, pricing model, pricing tiers, features, integrations, who it suits, who should skip it
- — re-verified summary, description, our verdict, our analysis, pricing model, pricing tiers, features, integrations, who it suits, who should skip it
- — re-verified summary, description, our verdict, our analysis, pricing model, pricing tiers, features, integrations, who it suits, who should skip it
- — re-verified summary, description, our verdict, our analysis, pricing model, pricing tiers, features, integrations, who it suits, who should skip it
- — re-verified summary, description, our verdict, our analysis, pricing model, pricing tiers, features, integrations, who it suits, who should skip it
Showing the 6 most recent of 17 verification passes.
Free to cite with attribution — this page re-verifies continuously.
12-month cost
Project the real annual outlay, including the implied monthly cost when only an annual tier is published.
Vendor list price only. Add-on usage, seat overages, and contract minimums are surfaced under Hidden costs & gotchas.
Plans compared
For each published LiteLLM tier: who it actually fits, and what it adds vs. the previous tier. Cross-reference the cost calculator above for projected annual outlay.
Open Source
$0/mo
Ideal for
Teams that want to self-host a free AI gateway and have the DevOps capacity to manage it; ideal for startups and mid-size companies with basic access control and spend tracking needs.
What this tier adds
Free entry point with virtual keys, teams, spend tracking, budgets, rate limits, fallbacks, request logging, and Prometheus metrics. No SSO, audit logs, or support.
Enterprise
Custom (Annual)
Ideal for
Enterprises running LiteLLM in production that need governance (SSO, audit logs), security integrations (secret managers), and 24/7 support with SLAs.
What this tier adds
Adds SSO + SCIM, OIDC/JWT auth, audit logs, secret managers, multi-region control plane, and 24/7 support. Priced by annual request capacity, never per token.
Where the pricing makes sense
The company stage and team size where LiteLLM's pricing actually pencils out — and where peers do it cheaper.
LiteLLM's free self-hosted tier is a no-brainer for any team that can manage its own infrastructure. Enterprise is priced by annual request capacity, which can be more cost-effective than per-token pricing from managed gateways like Portkey or Helicone, but only if you have the DevOps resources to run it.
Setup time & first value
How long it actually takes to get something useful out of LiteLLM — broken out by persona, not the marketing-page minute.
A basic self-hosted setup takes about 10-15 minutes using the quickstart. Adding SSO, audit logs, and routing rules to match your org can take a few hours. For a production-grade deployment with air-gapping and custom plugins, budget a day or two.
Switching to or from LiteLLM
How to bring data in from common predecessors and how to get it back out — written for the switcher, not the buyer.
- →From direct provider APIs: Point your OpenAI SDK to the LiteLLM proxy URL and keep the same model names; no code changes needed.
- →From a custom in-house proxy: Reuse your existing API keys and models by configuring them in LiteLLM's config file; migrate your routing logic to Auto Router or fallback rules.
- →From a managed gateway like Portkey or Helicone: Export your usage logs and configure virtual keys in LiteLLM; use the OpenAI-compatible endpoints to maintain compatibility.
- ↗To a managed gateway: If you want to offload operations, migrate to Portkey or Helicone by re-pointing your API base URL; your code stays the same.
- ↗To direct provider APIs: For simple apps, you can revert to calling OpenAI/Anthropic directly, but you lose cost tracking and fallbacks.
Integrations
Resources & Guides
Tutorials & Learning
Official links
Featured Head-to-Head Comparisons
Langfuse vs Litellm
If you need to centrally manage and route requests across 100+ LLMs with cost tracking and fallbacks, LiteLLM is your gateway. If you need deep observability, prompt management, and evaluations for production LLM apps, Langfuse is the observability layer. They integrate together, so a powerful stack uses both.
Langchain vs Litellm
If you’re building complex, multi-step agents and need deep observability and evaluation, LangChain is your pick. If you’re a platform team unifying access to many models with strict cost and access controls, LiteLLM is the straightforward choice. For most teams, they complement each other: use LangChain for agent logic, LiteLLM in front as the gateway.
Popular in LLM Gateways & Model Routers
OpenRouter Agents
One unified AI API for 500+ models, 80+ providers, pay-per-token without subscriptions.
Intrascope
Centralize access to ChatGPT, Claude, Gemini, and more with multi-model governance.
Frequently Asked Questions
Categories
Best-of guides
Topics
Used LiteLLM? Help shape our editorial sentiment research.