Features
Unified API for 1600+ LLMs across providers
Smart routing with fallbacks, load balancing, canary testing
Deterministic and LLM-based guardrails
Semantic and simple caching to reduce latency and cost
Observability with logs, traces, feedback, and alerts
Prompt management with versioning and playground
Virtual key management with rotation and usage monitoring
Automatic retries and request timeouts
Multimodal support (vision, audio, image generation)
File uploads and real-time API recording
MCP Gateway for agent governance
Agent Gateway for AI agent observability and control
Batch processing (provider batch APIs and custom batching)
Fine-tuning support via unified API
Role-based access control and SSO
Local model execution for offline inference
Connect online APIs: OpenAI (ChatGPT), Anthropic (Claude), Google (Gemini), Meta (Llama), Mistral AI, Alibaba (Qwen),
123+ HuggingFace models available
Memory feature (coming soon) for context persistence
Minimalist UI with dark mode
Open source under MIT license on GitHub
Cross-platform desktop app (Mac, Windows, Linux)
Version 0.8.3 with bug fixes and performance improvements
