Gateway

Portkey's pitch is consolidation: instead of wiring 1600+ provider integrations yourself, you call one OpenAI-compatible endpoint and let the gateway handle provider diversity. The reliability layer is where it justifies itself first. Conditional routing, automatic fallbacks during provider errors, load balancing, automatic retries, request timeouts, and canary testing cover the failure modes that otherwise wake you at 3am. Add simple and semantic caching with unlimited TTL stream-from-cache and you have two independent cost levers — fewer upstream calls, and fewer cold ones. The observability layer is the second reason teams stay: logs, traces, feedback, custom metadata, and filters across every LLM call, with cost attribution per use case. The March 2026 changelog tightened governance with vault-backed Secret References pulling from AWS Secrets Manager, Azure Key Vault, and HashiCorp Vault, weekly budget windows, and endpoint-scoped rate limits — the kind of controls auditors ask about. Guardrail coverage now spans LLM and partner guardrails plus Zscaler AI Guard, Akto Agentic Security, and Bedrock Guardrails customHost. The direction of travel in 2026 is agents: Portkey shipped a Skills Registry for agent skills, an Agent Gateway sitting in front of agent traffic, MCP governance guidance, and a blog argument that agent vulnerabilities — MCP tool-description injection, calendar-invite prompt injection, unauthorized tool calls, cost overruns — are trust-boundary failures rather than model failures. That framing is useful, though it also means Portkey is betting its roadmap on agent workloads materializing the way it expects. The honest constraints: the free Developer tier records 10k logs per month with 3-day log retention and 30-day metrics — exceeding the limit stops recording, though your requests are not blocked, so you can silently lose the audit trail you're relying on. Production at $49/month raises that to 100k logs, 30-day log retention, and 90-day metrics, then charges $9 per additional 100k. Advanced security and governance — SSO, custom guardrail hooks, custom retention, granular budgets — sit on the Enterprise tier with custom pricing. And adopting a gateway adds a hop: Portkey benchmarks 20–40ms of added latency versus direct API calls, offset by caching and routing when those apply. Where it fits: platform and platform-adjacent teams running multi-provider production traffic, organizations that need per-team virtual keys with budgets, and compliance-bound enterprises that need VPC hosting and BAAs. Where it doesn't: single-model prototypes, no-code builders, fully offline inference, and teams unwilling to put a hosted gateway in the request path. This tool is an infrastructure control plane, not a thin wrapper over someone else's model.