ngrok AI Gateway

The pitch here is honest: you almost certainly already call more than one model, and your current setup probably involves provider dashboards that tell you the total bill but not which app, developer, or model drove it. That's the gap the gateway fills. The strongest parts are small and concrete — changing baseURL to https://gateway.ngrok.ai and swapping the API key keeps your existing OpenAI or Anthropic SDK code intact, and model arrays like ["gpt-5.4", "claude-opus-4-6"] with a self-hosted model first let you express failover declaratively instead of writing retry logic. Scoped access keys solve a real, boring security problem: instead of one key that opens every provider, each developer or app gets its own key with an allowlist of providers and models. Bring-your-own-key means you keep your negotiated OpenAI and Anthropic rates; using ngrok keys passes inference through at cost. Observability rolls up tokens, latency, and errors across every call you route, which provider dashboards structurally cannot do. The weaknesses are equally clear. This is a managed cloud service — availability and data flow depend on ngrok's infrastructure, and there's no on-prem or self-hosted option described. Routing to local LLMs works over private connectivity, but the gateway itself remains ngrok-hosted. Advanced enterprise controls (SSO, audit logs) aren't on the published flat fee, so governance-heavy orgs will need to talk to sales. And the flat $0.05/M route fee is cheap at low volume but compounds at high token throughput, where a self-hosted LiteLLM or a cloud provider's native gateway may be cheaper. It fits teams that value speed of setup and don't want to run routing infrastructure; it doesn't fit teams with strict data-residency rules or unusually bespoke routing needs.