Features
Unified API for 1600+ LLMs across providers
Smart routing with fallbacks, load balancing, canary testing
Deterministic and LLM-based guardrails
Semantic and simple caching to reduce latency and cost
Observability with logs, traces, feedback, and alerts
Prompt management with versioning and playground
Virtual key management with rotation and usage monitoring
Automatic retries and request timeouts
Multimodal support (vision, audio, image generation)
File uploads and real-time API recording
MCP Gateway for agent governance
Agent Gateway for AI agent observability and control
Batch processing (provider batch APIs and custom batching)
Fine-tuning support via unified API
Role-based access control and SSO
Unified API for 400+ models across 70+ providers
Automatic failover routing when a provider goes down
Model Fusion: combine budget models to rival frontier performance
Subagent delegation: frontier models hand off tasks to cheaper workers
Speech/transcription APIs (May 2026)
Unified Image API with capability discovery across 30+ models (June 2026)
OpenAI SDK compatible out of the box
Server-side presets to survive model deprecation
Edge-deployed for minimal latency
Budget controls and spend limits
Credit-based pricing with no subscriptions
Model routing visualization for latency and cost
Private model support for enterprise deployments
Data policy-based routing and enforcement
MCP server for coding agents (June 2026)
