DeepInfra
DeepInfra runs serverless AI inference on 100+ open models through one OpenAI-compatible API key.
Open-weight inference at volume is where DeepInfra earns its keep, and the per-token numbers are the reason to look: $0.06/1M input on DeepSeek-V4-Flash-0731 and $0.02 on Llama-3.1-8B are hard to argue with. The 1024k-context flagships and cached-input discounts matter for RAG and agent pipelines that resend the same system prompt a thousand times. Bring your own training stack, and benchmark time-to-first-token on your own traffic before you commit.
Verified 1d ago · liveness 88/100 · cite: rightaichoice.com/tools/deepinfra
- Startups running high-volume open-weight LLM inference that need low cost per token
- Teams building RAG or long-context agent pipelines that benefit from cached input pricing
- Developers who want new open-weight models on release day without managing GPUs
- Enterprises with zero-retention and SOC 2 / ISO 27001 compliance requirements
- Teams that need custom model training — DeepInfra hosts models for inference, not training
- Organizations with data residency mandates outside US-based data centers
- Users who want a community model hub with hosted spaces and datasets
We scan live Reddit threads, YouTube comments, X posts, G2 reviews and other communities — and hand you an honest verdict in under a minute.
- Honest verdict, not marketing
- Real pros & cons from real users
- Attributed quotes with receipts
3 free scans · no card needed
Skip DeepInfra if you need a training or fine-tuning platform rather than an inference endpoint, or if data residency has to sit outside the United States.
Cached-input discounts only apply to the models that publish a cached rate — DeepSeek-V3, DeepSeek-V3.2 partially and several Qwen entries bill full input price on every call.
The per-token rates fit seed-stage and growth startups running open weights at volume — DeepSeek-V4-Flash-0731 at $0.06/1M in and GLM-5.3-Flash at $0.075/1M in undercut most managed inference rivals. Mid-size platform teams graduate to private A100–B300 deployments with autoscaling, while enterprises buy custom contracts with zero retention and SOC 2 / ISO 27001 US infrastructure. Kimi-K3 at $2.85/$14.25 per 1M and Qwen3.7-Max at $2.50/$7.50 sit at frontier-lab price levels for teams that need
In short
DeepInfra — DeepInfra runs serverless AI inference on 100+ open models through one OpenAI-compatible API key. Best for Startups running high-volume open-weight LLM inference that need low cost per token, Teams building RAG or long-context agent pipelines that benefit from cached input pricing, Developers who want new open-weight models on release day without managing GPUs. Plans from $0.02.
No head-to-head comparison for DeepInfra yet — the alternatives below are the closest matches we list.
What's new in DeepInfra
Checked 3 days agoAcross the latest 6 updates: 2 launches, 1 pricing change, 2 community discussions and 1 news mention.
Granite-4.2-8B Pricing Guide: Provider Comparison
Granite-4.2-8B priced at $0.06 per million input and $0.25 per million output tokens; scores 11 on Artificial Analysis Intelligence Index for small open-weight models.
IBM Granite 4.2 8B Now Available on DeepInfra
IBM Granite 4.2 8B added with three selectable reasoning modes: full chain-of-thought, low-effort shortcut, and direct-answer without reasoning.
Open-Weight Model Licenses: What to Check Before You Ship
DeepInfra outlines license checks enterprises should run before deploying open-weight models into production workloads.
NVIDIA Nemotron 3.5 Super VL on DeepInfra
NVIDIA Nemotron 3.5 Super VL available at launch via OpenAI-compatible API with pay-per-token pricing, combining vision, reasoning, and tool calling.
AI Spend Audit: How to Reduce Costs With Open Models
DeepInfra argues open-weight models are moving from experimentation into production as frontier-model costs rise, citing Tinder and other enterprises.
TypeSafe Jev Review: Testing 5 Jobs on a Non-Chat Model
DeepInfra reviews TypeSafe Jev, a classification-style model that returns decisions on input text and questions rather than generating prose.
What people actually say about DeepInfra — is it worth it?
We ran a structured research pass across product reviews, community discussions, and post-purchase forum threads to surface the patterns vendors won't publish themselves. Below: the recurring strengths, the hidden costs people mention most, and the cohort that consistently regrets adopting this tool.
5 mentions across 4 sources (Hacker News, YouTube, Stack Overflow, Lemmy), 49 more we could not attribute · researched Oct 8, 2026.
Weighted by the 54 posts each of 4 sources contributed.
- +DeepSeek-V4-Flash-0731 at $0.06/1M input and $0.015 cached is consistently the cheapest cited open-weight endpoint
- +Users repeatedly call it 'quite fast,' often beating OpenRouter-routed alternatives in real workloads
- +One OpenAI-compatible API key covers 100+ models including text, vision, embeddings, TTS, and video
- +Zero-retention policy plus SOC 2/ISO 27001 certification satisfies most compliance checklists out of the box
- +No peak-hour pricing — Lemmy users cite this as a decisive advantage over the official DeepSeek API
- −vLLM middleware bugs reportedly strip reasoning content from GLM models, silently degrading output quality
- −Image parsing on the OpenRouter route regressed between July and September 2026 per OpenRouter's own team
- −Community reputation for privacy is mixed — one HN commenter actively blocks the provider
- −Provider-level consistency isn't guaranteed; researchers dropped it for replication-sensitive work
- −Smaller model catalog than OpenRouter, which routes across multiple providers for fallback coverage
- • No free tier has been publicly documented in the community data — this is a paid product with no confirmed free trial
- • Cached input discounts only apply if you actually hit the cache window; cold calls pay full input price
- • Multimodal execution-time billing is opaque to estimate before you run the workload once
- • Private deployments on the GPU rental tiers add ops costs on top of the listed per-hour rate
Viability Score
How well maintained and how widely used is DeepInfra? Built from what the vendor actually publishes (docs, changelog, tutorials, integrations, pricing), whether the site is live, and how much real users discuss it. How we calculate this
Last calculated: October 2026
How we score →Key Features
- OpenAI-compatible API covering 100+ open and proprietary models through one key
- Anthropic SDK and Claude Code compatible endpoints alongside the OpenAI schema
- Per-token pricing with cached input discounts (DeepSeek-V4-Flash-0731: $0.06 in, $0.015 cached, $0.18 out)
- Execution-time billing for speech, image, music, video and world models with no idle charges
- 1024k-token context on Step-5-Preview, Hy4-preview, GLM-5.3, DeepSeek-V4.1-Flash, Kimi-K3 and MiMo-V2.6
- Xiaomi MiMo-V2.6-Pro and MiMo-V2.6-Flash: natively omnimodal across text, image, video and audio
- Tencent Hy4-preview: 770B MoE, 49B active, 1024k context, $0.834 in / $2.501 out per 1M
- Step-5-Preview: 600B MoE with 27B active, 1M context, text and image input for agentic work
- NVIDIA Nemotron-3.5-Super-VL: 120B MoE (12B active) for vision, documents, GUIs and tool use
- IBM Granite 4.2 8B with selectable reasoning modes (full chain-of-thought, low-effort, direct answer)
- Text-to-image with FLUX and Stable Diffusion plus LoRA; text-to-video and world models via NVIDIA Cosmos
- Text-to-speech, speech recognition, text-to-music, embeddings, rerankers and zero-shot image classification
- Vision and OCR via Qwen3-VL, DeepSeek-V4-Flash-Vision-Exp and Nemotron-3.5-Super-VL
- Private deployments of custom LLMs and LoRA adapters on A100 through B300 with autoscaling
- Zero-retention policy on inputs, outputs and user data; SOC 2 and ISO 27001 certified
About DeepInfra
DeepInfra is a serverless inference cloud for developers who'd rather call an API than rent GPUs. One OpenAI-compatible key reaches 100+ models across text generation, automatic speech recognition, embeddings, rerankers, text-to-image, text-to-music, text-to-speech, text-to-video, world models and zero-shot image classification, plus Claude and Gemini family endpoints and Anthropic SDK / Claude Code compatibility. Billing splits two ways. Language models charge per token with cached-input discounts, while most non-text models bill by inference execution time, so idle capacity costs nothing. The floor is low: DeepSeek-V4-Flash-0731 lists $0.06 per 1M input ($0.015 cached) against $0.18 out, and Meta-Llama-3.1-8B-Instruct-Turbo sits at $0.02 in / $0.04 out. Discounts stack on top — DeepSeek-V4.1-Flash-1017 is currently 51% off. Long context is the other hook. Step-5-Preview (600B total, 27B active), Tencent's Hy4-preview (770B MoE, 49B active), Kimi-K3, GLM-5.3, DeepSeek-V4.1-Flash and both Xiaomi MiMo-V2.6 models carry 1024k windows. MiMo-V2.6-Pro (1.02T total, 42B active) and MiMo-V2.6-Flash are natively omnimodal over text, image, video and audio. NVIDIA Nemotron-3.5-Super-VL, a 120B MoE with 12B active, adds vision, document and GUI understanding with tool calling. Infrastructure is in-house and US-based, backed by SOC 2 and ISO 27001 certification with a zero-retention policy on inputs, outputs and user data. Teams that outgrow serverless can rent on-demand NVIDIA DGX B300 at $5.89 per instance-hour or deploy private fine-tuned LLMs and LoRA adapters with autoscaling. Position it as the price list to beat for open-weight inference — not a training platform.
Behind the Verdict
We'd reach for DeepInfra when the bill is the problem. If you're pushing millions of tokens a month through open-weight models, the per-token list here — DeepSeek-V4-Flash-0731 at $0.06 in / $0.18 out, Llama-3.1-8B-Turbo at $0.02 / $0.04 — is the first quote to hold against Together AI and Fireworks. Cached input pricing is the quiet win for RAG: a fat system prompt resent on every call drops to $0.015 per 1M. Long-context work is the second reason. Ten models on the price sheet carry 1024k windows, and the omnimodal MiMo-V2.6 pair handles text, image, video and audio without chaining three vendors. For agent pipelines that need frontier-ish reasoning at a discount, Step-5-Preview at $1.00 in / $2.70 out is the value pick; Kimi-K3 at $2.85 / $14.25 is the expensive one and you should know why you're paying it. The catalog moves fast. DeepSeek-V4.1-Flash-1017, NVIDIA Nemotron-3.5-Super-VL and IBM Granite 4.2 8B all landed recently, and the $107M Series B is the funding behind that cadence. If your roadmap depends on new open-weight releases appearing on day one, this is a sensible stable of models. Where it bites: there's no training. DeepInfra hosts models for inference, so fine-tuning happens elsewhere and you bring the artifacts — LoRA adapters deploy privately on A100 through B300 if you need them. Data stays in US data centers, which is fine for most teams and a blocker for anyone with EU residency clauses in a customer contract. Latency is the unquantified risk. Execution-time billing for speech, image and video means you pay for compute used, but published token prices tell you nothing about time-to-first-token under your concurrency. Run a real load test before you migrate a consumer-facing path. Against a raw GPU rental, DeepInfra wins when utilization is
Researching DeepInfra? Get your full AI stack in 60 seconds.
Free, no signup — tell us your goal and get tools matched to your budget & existing stack.
Real-world workflow fit
Concrete scenarios for the personas DeepInfra actually fits — and what changes day-one when you adopt it.
Swaps the base URL in an existing OpenAI SDK script to api.deepinfra.com/v1/openai, sets the model to deepseek-ai/DeepSeek-V4-Flash-0731, and pulls a key from the dashboard.
Outcome: First API call returns without a code rewrite, and input tokens bill at $0.06 per 1M with $0.015 cached for repeat system prompts.
Indexes documents with DeepInfra embedding models, adds a reranker for precision, and routes generation to GLM-5.3-Flash while the 50% promotional discount holds.
Outcome: Retrieval and generation both run under one API key, with cached-input pricing trimming repeat-prefix cost on long retrieved contexts.
Deploys a custom LLM and LoRA adapter to a private autoscaling endpoint on A100/H100/H200/B200/B300 and points internal apps at the private URL.
Outcome: The model runs on dedicated hardware with zero retention on inputs, outputs and user data, inside SOC 2 and ISO 27001 certified US data centers.
Use Cases
- Change one OpenAI SDK base URL and ship a chatbot on DeepSeek-V4-Flash-0731 at $0.06 per 1M input tokens
- Run vision and OCR on scanned documents with Qwen3-VL or DeepSeek-V4-Flash-Vision-Exp
- Build a RAG pipeline on embeddings and rerankers with cached input cutting repeat-prefix costs
- Generate images at scale through the FLUX or Stable Diffusion APIs, including image LoRA adapters
- Serve long-horizon coding agents on Kimi-K3 or MiMo-V2.6-Pro with 1024k context and tool calling
- Split agent work across two models — a cheap one for mechanical steps, a frontier one for reasoning
- Batch asynchronous jobs through the Batch API and Files API instead of holding a request open
- Deploy a fine-tuned LLM or LoRA adapter on a private autoscaling endpoint for data isolation
Models Under the Hood
as of 2026-10-22
Limitations
- DeepInfra is a serverless inference host, not a training service: its catalog covers 100+ open and some proprietary models, and you can only run models it supports.
- Billing is mixed — most language models are per-token with cached-input discounts (e.g.
- DeepSeek-V4-Flash-0731 at $0.06 in / $0.015 cached / $0.18 out per 1M), while most speech, image, music, video and world models are billed by inference execution time.
- Context windows reach up to 1024k on flagship text models (DeepSeek-V4-Flash-0731, DeepSeek-V4-Pro, Kimi-K3) but fall to 160k on older DeepSeek text models such as DeepSeek-V3.1 and DeepSeek-R1-0528.
- Promotional discounts rotate frequently, so a discount tag today may not hold later.
as of 2026-10-10
Verification history
We have re-verified DeepInfra 21 times since . Each pass re-reads the vendor's own pages and re-checks every listed field against that evidence; passes where nothing had changed are marked as such.
- — re-verified summary, description, our verdict, our analysis, pricing model, pricing tiers, features, integrations, who it suits, who should skip it
- — re-verified summary, description, our verdict, our analysis, pricing model, pricing tiers, features, integrations, who it suits, who should skip it
- — re-verified summary, description, our verdict, our analysis, pricing model, pricing tiers, features, integrations, who it suits, who should skip it
- — re-verified summary, description, our verdict, our analysis, pricing model, pricing tiers, features, integrations, who it suits, who should skip it
- — re-verified summary, description, our verdict, our analysis, pricing model, pricing tiers, features, integrations, who it suits, who should skip it
- — re-verified summary, description, our verdict, our analysis, pricing model, pricing tiers, features, integrations, who it suits, who should skip it
Showing the 6 most recent of 21 verification passes.
Free to cite with attribution — this page re-verifies continuously.
12-month cost
Project the real annual outlay, including the implied monthly cost when only an annual tier is published.
Vendor list price only. Add-on usage, seat overages, and contract minimums are surfaced under Hidden costs & gotchas.
Plans compared
For each published DeepInfra tier: who it actually fits, and what it adds vs. the previous tier. Cross-reference the cost calculator above for projected annual outlay.
Pay-as-you-go — text generation (per token)
$0.02–$2.85 / 1M input tokens
Pay-as-you-go — non-text models (execution time)
Billed per second of inference
On-Demand GPU (DGX B300)
$5.89 / instance-hour
Ideal for
Engineers who need Blackwell-class hardware briefly for training runs or custom workloads but do not want a reserved capacity commitment.
What this tier adds
Adds raw NVIDIA DGX B300 instances at $4.89 per instance-hour with hourly billing and SSH access.
Private Deployment (custom LLMs & LoRA)
Custom
Where the pricing makes sense
The company stage and team size where DeepInfra's pricing actually pencils out — and where peers do it cheaper.
The per-token rates fit seed-stage and growth startups running open weights at volume — DeepSeek-V4-Flash-0731 at $0.06/1M in and GLM-5.3-Flash at $0.075/1M in undercut most managed inference rivals. Mid-size platform teams graduate to private A100–B300 deployments with autoscaling, while enterprises buy custom contracts with zero retention and SOC 2 / ISO 27001 US infrastructure. Kimi-K3 at $2.85/$14.25 per 1M and Qwen3.7-Max at $2.50/$7.50 sit at frontier-lab price levels for teams that need
Setup time & first value
How long it actually takes to get something useful out of DeepInfra — broken out by persona, not the marketing-page minute.
A developer can be making a first API call in about a minute: create a key in the dashboard, change the base URL to api.deepinfra.com/v1/openai, and the docs publish a no-install Quickstart. Wiring a full RAG or agent stack — embeddings, reranker, caching, batch jobs — is typically an afternoon. Private deployments and DGX B300 rental take longer because capacity has to be provisioned and
Switching to or from DeepInfra
How to bring data in from common predecessors and how to get it back out — written for the switcher, not the buyer.
- →From OpenAI API: change the base URL to api.deepinfra.com/v1/openai, reuse the OpenAI SDK and pick a DeepSeek, GLM, Kimi or Qwen model.
- →From Together AI or Fireworks: port the same OpenAI-style calls over and re-benchmark latency, since the model catalog overlaps heavily.
- →From self-hosted vLLM or TGI: move token generation to the serverless endpoint and keep GPUs only for private or batch workloads.
- →From Anthropic API: DeepInfra publishes Anthropic SDK and Claude Code compatible endpoints so existing Claude client code can point at its models.
- ↗To a frontier lab API: move back to OpenAI or Anthropic when you need hosted closed weights the DeepInfra catalog does not carry.
- ↗To a training platform: shift to a service that handles fine-tuning and experiment tracking, since DeepInfra hosts models for inference only.
- ↗To a hyperscaler endpoint: migrate to AWS Bedrock or Vertex AI when procurement or data residency requires your existing cloud contract.
Integrations
Resources & Guides
- Documentationdeepinfra.com
What is DeepInfra
AI inference cloud — OpenAI-compatible API, 100s of open-source models, private GPU deployments, and GPU rental.
- Resourcedeepinfra.com
Blog | Fast & Reliable AI Inference
Discover the latest machine learning models and infrastructure! Learn how to enhance your AI applications, and more!
- Quickstartdeepinfra.com
What is DeepInfra
AI inference cloud — OpenAI-compatible API, 100s of open-source models, private GPU deployments, and GPU rental.
- Documentationdeepinfra.com
What is DeepInfra
AI inference cloud — OpenAI-compatible API, 100s of open-source models, private GPU deployments, and GPU rental.
- Documentationdeepinfra.com
What is DeepInfra
AI inference cloud — OpenAI-compatible API, 100s of open-source models, private GPU deployments, and GPU rental.
- Documentationdeepinfra.com
What is DeepInfra
AI inference cloud — OpenAI-compatible API, 100s of open-source models, private GPU deployments, and GPU rental.
- Documentationdeepinfra.com
What is DeepInfra
AI inference cloud — OpenAI-compatible API, 100s of open-source models, private GPU deployments, and GPU rental.
- Documentationdeepinfra.com
What is DeepInfra
AI inference cloud — OpenAI-compatible API, 100s of open-source models, private GPU deployments, and GPU rental.
- Documentationdeepinfra.com
GPU Instances
Rent dedicated B200 GPU instances with SSH access for training, fine-tuning, and custom workloads.
- Documentationdeepinfra.com
Anthropic SDK & Claude Code
Use DeepInfra models with the Anthropic Messages API, Claude Code, and the Anthropic SDK.
Tutorials & Learning
YouTube returned 6 videos for “DeepInfra”, and we withheld 6: 6 could not be judged, because “DeepInfra” is a single word that other videos use for other things. We are showing none, because we could not prove any of them are about DeepInfra.
Official links
Tools that pair well with DeepInfra
Common stack mates teams adopt alongside DeepInfra, with the specific reason each pairing earns its keep.
LocalAI
LocalAI is an MIT-licensed local AI runtime that serves text, voice, vision, image and agent workloads behind an OpenAI-compatible API.
Inference Engine by GMI Cloud
GMI Cloud Inference Engine runs frontier text, image, video, and audio models behind one OpenAI-compatible API on dedicated NVIDIA GPUs.
novita.ai
Novita AI runs 200+ open-weight LLMs, image, audio and video models through one serverless API, plus GPU cloud and an Agent Sandbox.
Alternatives to DeepInfra
View all
LocalAI
LocalAI is an MIT-licensed local AI runtime that serves text, voice, vision, image and agent workloads behind an OpenAI-compatible API.
Inference Engine by GMI Cloud
GMI Cloud Inference Engine runs frontier text, image, video, and audio models behind one OpenAI-compatible API on dedicated NVIDIA GPUs.
Frequently Asked Questions
Categories
Best-of guides
Used DeepInfra? Help shape our editorial sentiment research.