LiteLLM
LiteLLM is a self-hosted AI gateway putting 140+ providers and 1,800+ models behind one OpenAI-compatible API.
For a platform team running three or more providers in production, LiteLLM is the most practical self-hosted answer: the OSS tier gives you virtual keys, per-tag budgets, fallbacks, and Auto Router v2 without a license fee, and day-0 model support means Grok 4.7 or Qwen3.8-Omni-Flash appear without an app change. The catch is that you operate it — Postgres, upgrades, and on-call are yours. Enterprise (annual, priced by request capacity) is where SSO + SCIM, audit logs, and RBAC live. If you want managed, look at Portkey, Kong, or OpenRouter.
Verified 2d ago · liveness 81/100 · cite: rightaichoice.com/tools/litellm
- Platform teams giving the whole org one key to 140+ models, MCP servers, and agents
- Organizations enforcing per-team budgets with tag-level caps on AI spend
- Multi-provider production stacks needing fallbacks, load balancing, and router groups
- Regulated enterprises requiring self-hosting, air-gapped deployment, SSO/SCIM, and audit logs
- Single-provider shops that gain nothing from multi-model abstraction or routing
- Teams with no appetite to own Postgres, patching, and gateway on-call
- Small projects where a direct provider SDK is simply less work
We scan live Reddit threads, YouTube comments, X posts, G2 reviews and other communities — and hand you an honest verdict in under a minute.
- Honest verdict, not marketing
- Real pros & cons from real users
- Attributed quotes with receipts
3 free scans · no card needed
Skip LiteLLM if you want someone else to run your gateway — self-hosting, Postgres, upgrades, and on-call are the whole premise, and the SSO, audit-log, and RBAC governance you may need is an annual Enterprise contract rather than a self-serve upgrade.
SSO + SCIM, audit logs, secret managers with key rotation, and RBAC with org and team admins are Enterprise-only, so security-conscious teams can't stay on the free tier once an audit lands.
Self-hosted OSS at $0 with no license fee is cheaper than any managed gateway at low volume — Portkey and Kong charge for their hosted control plane, and OpenRouter takes a margin per hosted call. Where LiteLLM's economics flip: once you need SSO, audit logs, and RBAC, the annual Enterprise quote sized to request capacity can exceed a per-seat managed plan, so compare the Enterprise number against Portkey or Kong at your actual seat count before you sign.
In short
LiteLLM — LiteLLM is a self-hosted AI gateway putting 140+ providers and 1,800+ models behind one OpenAI-compatible API. Best for Platform teams giving the whole org one key to 140+ models, MCP servers, and agents, Organizations enforcing per-team budgets with tag-level caps on AI spend, Multi-provider production stacks needing fallbacks, load balancing, and router groups. Free to use.
What's new in LiteLLM
Checked 2 days agoAcross the latest 5 updates: 3 feature updates and 2 launches.
Day 0 support for Grok 4.7
Grok 4.7 became available through LiteLLM on its release day, priced the same as Grok 4.6.
Claude Code server-side auto mode support through LiteLLM
LiteLLM's native /v1/messages route forwards Anthropic's server-side Claude Code auto mode safeguards, with the fix shipping in the September 22 dev release.
TypeSafe Jev lands in LiteLLM v1.103.0-rc
TypeSafe AI's Jev classifier is callable through the LiteLLM proxy with logging and cost tracking in v1.103.0-rc.
Day 0 support for Qwen3.8-Omni-Flash
Qwen3.8-Omni-Flash is supported through LiteLLM with text, image, audio, and video input.
Auto Router adds five cost, speed and team-control improvements
The Auto Router update adds cache-cost estimates, Fast Mode by tier, team-managed routers, live CLI stats, and more accurate Anthropic savings estimates.
What people actually say about LiteLLM — is it worth it?
We ran a structured research pass across product reviews, community discussions, and post-purchase forum threads to surface the patterns vendors won't publish themselves. Below: the recurring strengths, the hidden costs people mention most, and the cohort that consistently regrets adopting this tool.
6 mentions across 5 sources (Hacker News, Product Hunt, Stack Overflow, GitHub, Lemmy), 47 more we could not attribute · researched Oct 10, 2026.
Weighted by the 53 posts each of 5 sources contributed.
- +One OpenAI-compatible API across 140+ providers and 1,800+ models cuts integration work dramatically
- +Free self-hosted tier includes virtual keys, teams, spend tracking, budget caps and rate limits
- +Day-0 support for new models like Grok 4.7 and Qwen3.8-Omni-Flash is a genuine differentiator
- +Community clearly runs it in real production — VPS hobbyists, enterprises, and platforms embedding it
- +Auto Router v2 with complexity/semantic/adaptive signals plus Router Plugins for custom routing
- −Repeated high-severity CVEs including an actively exploited pre-auth SQLi raise real deployment risk
- −Security incident coverage touched thousands of orgs — trust damage lingers even where attribution shifted
- −Large footprint (700MB, 100k+ LOC) pushed users to build a de-bloated fork just to escape it
- −5,408 open GitHub issues signals a backlog that can outpace triage on security-sensitive bugs
- −Users report churning to leanroute.dev after CVE exposure, which argues against long-term lock-in
- • You pay in engineering time for patching, hardening and upgrades the free tier pushes onto you
- • Self-hosting means infrastructure, monitoring and on-call costs are yours
- • Enterprise annual commitment is tied to request capacity, which can grow with usage
Viability Score
How well maintained and how widely used is LiteLLM? Built from what the vendor actually publishes (docs, changelog, tutorials, integrations, pricing), whether the site is live, and how much real users discuss it. How we calculate this
Last calculated: October 2026
How we score →Key Features
- One OpenAI-compatible API across 140+ providers and 1,800+ models
- Day-0 support for new models: Grok 4.7 (2026-09-21) and Qwen3.8-Omni-Flash (2026-09-18)
- Native /v1/messages route with Anthropic Claude Code server-side auto mode safeguards
- Multimodal input support: text, image, audio, and video (Qwen3.8-Omni-Flash)
- Auto Router v2 with complexity, semantic, and adaptive routing signals
- Auto Router cache-cost estimates, Fast Mode by tier, team-managed routers, live CLI stats
- Router plugins for custom routing signals and shadow evaluations
- Virtual keys scoped by user, team, project, and app
- Spend tracking by key, user, team, and organization
- Budget caps including per-tag budgets for prod, staging, and eval
- RPM/TPM rate limits with cooldowns and max monthly budgets
- LLM fallbacks and load balancing across model deployments
- MCP Gateway and Agent Gateway (A2A) for governed tool and agent access
- Guardrails: Presidio PII masking, Lakera prompt injection, Aporia policy
- Observability via Prometheus, Langfuse, and OpenTelemetry
About LiteLLM
LiteLLM is a self-hosted AI gateway that sits between your applications and every model you use. One OpenAI-compatible API fronts 140+ providers and 1,800+ models, and the same gateway also reaches the MCP servers and A2A agents your teams run. A developer points at gpt-5, claude-opus-4-8, gemini-2.5-pro, bedrock/llama-4-maverick, or your own fine-tune without a fresh procurement cycle each time. Day-0 support is the pattern the vendor advertises and the blog backs up: Grok 4.7 landed on release day on 2026-09-21, and Qwen3.8-Omni-Flash (text, image, audio, and video input) arrived 2026-09-18. The native /v1/messages route forwards Anthropic's server-side Claude Code auto mode safeguards, with the fix shipping in the September 22 dev release. The open-source tier is free to self-host, forever, with no license fee. You get virtual keys scoped by team, project, or app; spend tracked by key, user, team, and organization; budget caps including per-tag budgets for prod, staging, and eval; RPM/TPM rate limits with cooldowns; fallbacks and load balancing across deployments; request logging; and Prometheus metrics. Auto Router v2 (updated 2026-09-17) adds cache-cost estimates, Fast Mode by tier, team-managed routers, live CLI stats, and tighter Anthropic savings estimates, plus router plugins for custom signals and shadow evaluations on sampled production traffic. Enterprise layers governance on the same binary: SSO + SCIM, OIDC/JWT auth, audit logs, secret managers with key rotation, RBAC with org and team admins, a multi-region control plane, and air-gapped deployment. Pricing is sized to annual gateway request capacity, deployment architecture, and support needs — never per token — and a 30-day Enterprise trial key arrives by email without a credit card. Named users on the vendor page include Netflix, NVIDIA, Okta, AT&T, and Lemonade; AT&T's quote cites some advanced AI task costs falling by as much as 56%. It is a gateway, not an agent workflow builder, and you operate it.
Behind the Verdict
LiteLLM's pitch is narrow and honest: put every model, MCP server, and agent behind one key, then see, cap, and route the spend. The vendor page says 140+ providers and 1,800+ models; the docs claim 100+ in the SDK quickstart. Either way, the value is that swapping a backend is a config change, not a rewrite. Okta's quoted architect says it plainly: switching the backend model is a gateway config update with no code changes, procurement cycles, or repetitive security reviews. Strengths that hold up under scrutiny. Spend control is granular to the level finance actually asks about — per key, per user, per team, per organization, with per-tag budgets so prod, staging, and eval are capped separately. The screenshot on the homepage shows budget bars like $18.3K / $40K by team, plus rate limits (TPM 2M / RPM 10K) and guardrails (PII masking, prompt-injection, secret detection) attached at the team level. Auto Router v2's September 2026 update added cache-cost estimates, Fast Mode by tier, team-managed routers, live CLI stats, and more accurate Anthropic savings estimates — routing that can be argued about with numbers rather than vibes, and shadow evaluations let you test a routing change on sampled production traffic first. Day-0 coverage is not marketing: Grok 4.7 shipped on LiteLLM the day it released (2026-09-21) at the same price as Grok 4.6, and Qwen3.8-Omni-Flash on 2026-09-18. The native /v1/messages route carries Claude Code server-side auto mode safeguards, which matters if your engineers run Claude Code through the gateway. Beyond LLMs, the same gateway governs MCP servers and A2A agents via the MCP Gateway and Agent Gateway, so tool access shares the key and audit trail. Weaknesses you should price in before you commit. You run it. Postgres, container upgrades, secrets, and gateway on-call are your team's job, and the OSS tier is community-supported. The governance features that make a gateway defensible in an audit — SSO + SCIM, OIDC/JWT, audit logs, secret managers with key rotation, org and team admins under RBAC, multi-region control plane — are Enterprise-only, sold annually and sized to annual request capacity, so it is a quote, not a checkout. The vendor does hand you a 30-day Enterprise trial key by email with no credit card, which is the right way to evaluate those features before a procurement cycle. Air-gapped deployment is available on both Enterprise Standard and SCALE. And know what it is not: an agent workflow builder or prompt-chaining framework. If you want a managed gateway with no servers to own, Portkey, Kong, or Cloudflare AI Gateway are the shape to compare; OpenRouter is hosted with no self-hosting path, which is the opposite trade. Where it fits: multi-provider production stacks, regulated enterprises that need self-hosting or air-gap, and platform teams tired of opening a new vendor review for every model. Where it doesn't: single-provider shops, teams unwilling to own infrastructure, and weekend prototypes
Researching LiteLLM? Get your full AI stack in 60 seconds.
Free, no signup — tell us your goal and get tools matched to your budget & existing stack.
Real-world workflow fit
Concrete scenarios for the personas LiteLLM actually fits — and what changes day-one when you adopt it.
Install the gateway with the one-line quickstart, connect OpenAI, Anthropic, and Vertex AI, then create one virtual key per team with model access scoped per project.
Outcome: The whole org reaches gpt-5, claude-opus-4-8, and gemini-2.5-pro through one key and one login in minutes instead of a procurement cycle per model.
Set per-team budgets and per-tag caps for prod, staging, and eval, then watch spend by key, user, team, and organization in the dashboard while setting RPM/TPM limits on the noisiest team.
Outcome: Spend is capped before it runs; the platform-eng team's $18.3K against a $40K budget is visible without exporting provider invoices.
Run the 30-day Enterprise trial key from the Enterprise page, wire up SSO + SCIM and audit logs, then test an air-gapped deployment on Enterprise Standard.
Outcome: Governance is proven in a trial before procurement, and the same gateway binary carries production with a multi-region control plane.
Use Cases
- Give each team one virtual key with model access scoped per project instead of per developer.
- Cap spend before it runs with per-team budgets, per-tag caps, and RPM/TPM limits in one dashboard.
- Route each request to the model that should handle it with Auto Router v2 and cut advanced-task costs.
- Swap an Anthropic backend for Bedrock or a self-hosted fine-tune with a config change, not a code rewrite.
- Reach MCP servers and A2A agents through the same gateway key your LLMs already use.
- Test routing changes on sampled production traffic with shadow evaluations before rolling them out.
- Add a new model the day it ships — Grok 4.7 and Qwen3.8-Omni-Flash both landed day-0.
- Centralize PII masking and prompt-injection checks at the gateway instead of in every app.
Models Under the Hood
as of 2026-10-16
Limitations
- LiteLLM is self-hosted, so your team owns deployment, patching, and monitoring, and the open-source tier is community-supported.
- The governance an audit usually asks for — SSO + SCIM, OIDC/JWT auth, audit logs, secret managers with key rotation, org and team admins under RBAC, and the multi-region control plane — is Enterprise-only, sold on an annual term sized to annual gateway request capacity, deployment architecture, and support needs, so it comes as a quote rather than a self-serve purchase.
- The gateway routes and governs requests; it is not an agent workflow builder or prompt-chaining framework.
- Air-gapped deployment is available on both Enterprise Standard and SCALE.
- The homepage names 140+ providers and 1,800+ models while the docs quickstart says 100+; check the Models page for the exact list you need.
as of 2026-10-20
Verification history
We have re-verified LiteLLM 20 times since . Each pass re-reads the vendor's own pages and re-checks every listed field against that evidence; passes where nothing had changed are marked as such.
- — re-verified summary, description, our verdict, our analysis, pricing model, pricing tiers, features, integrations, who it suits, who should skip it
- — re-verified summary, description, our verdict, our analysis, pricing model, pricing tiers, features, integrations, who it suits, who should skip it
- — re-verified summary, description, our verdict, our analysis, pricing model, pricing tiers, features, integrations, who it suits, who should skip it
- — re-verified summary, description, our verdict, our analysis, pricing model, pricing tiers, features, integrations, who it suits, who should skip it
- — re-verified summary, description, our verdict, our analysis, pricing model, pricing tiers, features, integrations, who it suits, who should skip it
- — re-verified summary, description, our verdict, our analysis, pricing model, pricing tiers, features, integrations, who it suits, who should skip it
Showing the 6 most recent of 20 verification passes.
Free to cite with attribution — this page re-verifies continuously.
12-month cost
Project the real annual outlay, including the implied monthly cost when only an annual tier is published.
Vendor list price only. Add-on usage, seat overages, and contract minimums are surfaced under Hidden costs & gotchas.
Plans compared
For each published LiteLLM tier: who it actually fits, and what it adds vs. the previous tier. Cross-reference the cost calculator above for projected annual outlay.
Open Source
$0
Ideal for
Platform teams self-hosting a multi-provider gateway who can own Postgres, upgrades, and on-call themselves
What this tier adds
Free entry point, no license fee, forever: 100+ providers on one OpenAI API, virtual keys, spend tracking, budgets, rate limits, fallbacks, logging, and Prometheus metrics.
Enterprise
Custom
Ideal for
Regulated or enterprise-wide deployments that need SSO, audit logs, air-gap, and support SLAs
What this tier adds
Adds SSO + SCIM, OIDC/JWT, audit logs, secret managers with key rotation, org and team RBAC, multi-region control plane, air-gap, and up to 24/7 support — annual, sized to request capacity.
Where the pricing makes sense
The company stage and team size where LiteLLM's pricing actually pencils out — and where peers do it cheaper.
Self-hosted OSS at $0 with no license fee is cheaper than any managed gateway at low volume — Portkey and Kong charge for their hosted control plane, and OpenRouter takes a margin per hosted call. Where LiteLLM's economics flip: once you need SSO, audit logs, and RBAC, the annual Enterprise quote sized to request capacity can exceed a per-seat managed plan, so compare the Enterprise number against Portkey or Kong at your actual seat count before you sign.
Setup time & first value
How long it actually takes to get something useful out of LiteLLM — broken out by persona, not the marketing-page minute.
Platform engineer: the container quickstart command gets a working gateway with an admin UI in minutes, first value same day. FinOps: budgets and per-tag caps are an afternoon once keys exist. Enterprise architect: SSO + SCIM, audit logs, and an air-gapped deployment are a multi-week rollout — start with the 30-day trial key, no credit card needed.
Switching to or from LiteLLM
How to bring data in from common predecessors and how to get it back out — written for the switcher, not the buyer.
- →From per-provider SDK calls: point your base URL at the LiteLLM gateway and keep the OpenAI-format calls you already wrote.
- →From OpenRouter: replicate the hosted routing on your own infrastructure and keep the keys, spend, and audit trail inside your network.
- →From a single Anthropic direct integration: switch the backend to Bedrock or a self-hosted model with a config change rather than a code rewrite.
- →From raw provider keys handed to developers: replace them with scoped virtual keys so spend and access are tracked per team.
- ↗To Portkey: export your virtual key and budget structure and rebuild it in a managed control plane if you no longer want gateway on-call.
- ↗To Kong AI Gateway: move routing and rate limits into an API-platform-native gateway if you already run Kong for the rest of your traffic.
- ↗To OpenRouter: surrender self-hosting for a hosted API if data residency and air-gap are not requirements.
Integrations
Resources & Guides
Tutorials & Learning
YouTube returned 6 videos for “LiteLLM”, and we withheld 6: 6 could not be judged, because “LiteLLM” is a single word that other videos use for other things. We are showing none, because we could not prove any of them are about LiteLLM.
Official links
Tools that pair well with LiteLLM
Common stack mates teams adopt alongside LiteLLM, with the specific reason each pairing earns its keep.
Featured Head-to-Head Comparisons
Langfuse vs Litellm
Langchain vs Litellm
If you’re building complex, multi-step agents and need deep observability and evaluation, LangChain is your pick. If you’re a platform team unifying access to many models with strict cost and access controls, LiteLLM is the straightforward choice. For most teams, they complement each other: use LangChain for agent logic, LiteLLM in front as the gateway.
Alternatives to LiteLLM
View all
Popular in LLM Gateways & Model Routers
OpenRouter Agents
One OpenAI-compatible API key that routes your requests across 500+ text, image, video, and audio models from 80+ providers.
Intrascope
Intrascope is a multi-model AI chat control layer with shared company context, model access governance and per-model budget caps.
Frequently Asked Questions
Categories
Best-of guides
Topics
Used LiteLLM? Help shape our editorial sentiment research.