Proxy
Local LLM proxy that meters every agent request and caps runaway spend before the bill does.
If you run coding agents and have ever found a runaway loop on your bill, this is the cheapest insurance available: MIT-licensed, local, one npm install, and the observation layer is genuinely in production rather than a demo. The shipped guards are unusually concrete — token-explosion detection, sliding-window velocity spikes, hard caps with configurable block/downgrade/warn actions, and a working kill switch. Buy the local product on its own merits; the hosted Team and Scale tiers are
Verified 4d ago · liveness 77/100 · cite: rightaichoice.com/tools/proxy
- Developers running Claude Code, Cursor, or OpenClaw who want per-request routing without touching code
- Small teams that need hard budget caps and runaway-loop detection on autonomous pipelines
- Engineers who want cost-per-run attribution from a local SQLite ledger they can export
- Privacy-sensitive shops that refuse to route LLM traffic through a hosted gateway
- Teams needing one dashboard merging runs across CI, laptops, and servers — that hosted layer is waitlisted, not built
- Non-technical buyers uncomfortable with a global npm install and setting ANTHROPIC_BASE_URL or OPENAI_BASE_URL
- Enterprises requiring SSO, RBAC, or a vendor support SLA — none are published for the local product
We scan live Reddit threads, YouTube comments, X posts, G2 reviews and other communities — and hand you an honest verdict in under a minute.
- Honest verdict, not marketing
- Real pros & cons from real users
- Attributed quotes with receipts
3 free scans · no card needed
Skip RelayPlane if you need a shared team dashboard, cross-machine run history, or SSO today — the hosted Team and Scale tiers ($49/mo and $199/mo) are an unbuilt early-access waitlist, and everything shipping is local-only.
The hosted Team and Scale tiers are waitlist-only demand tests — joining costs nothing, and there is no charge until the vendor decides to build them.
The local product is free forever under MIT, which undercuts every hosted gateway in the category on price and every managed alternative on data residency. The published Team tier is $49/mo and Scale is $199/mo, both early-access waitlist only with no card required. Against Helicone, Langfuse, or LiteLLM Cloud, RelayPlane's honest pitch is that you pay nothing for the local layer and only consider $49/mo if and when the cross-machine team features actually ship.
In short
Proxy — Local LLM proxy that meters every agent request and caps runaway spend before the bill does. Best for Developers running Claude Code, Cursor, or OpenClaw who want per-request routing without touching code, Small teams that need hard budget caps and runaway-loop detection on autonomous pipelines, Engineers who want cost-per-run attribution from a local SQLite ledger they can export. Free to start; paid plans from $49/mo.
What's new in Proxy
Checked 4 days agoAcross the latest 5 updates: 1 feature update, 1 changelog entry and 3 news mentions.
RelayPlane compares Helicone and Langfuse for LLM observability
Vendor post weighs proxy logging against OTel span tracing, covers pricing and the Helicone maintenance-mode question, and positions RelayPlane's cost-attribution angle.
RelayPlane on stopping runaway AI agent spend
Outlines three stackable spend caps: max_tokens and turn limits, session cost checks, and a hard dollar ceiling enforced at the proxy.
RelayPlane compares LiteLLM and OpenRouter
Vendor comparison of self-hosted proxy governance versus managed marketplace routing, covering hosting, cost, model coverage, and latency.
v1.9.39 ships live cost ticker CLI
`relayplane watch` renders today's spend, current session, model distribution, and the last 5 requests in a redraw-in-place terminal UI, backed by a new local /v1/stats/live endpoint polled every 3 seconds.
v1.9.36 quarantines failing credentials
TokenPool crediting quarantine evicts a credential from rotation after 2 consecutive 401s, so the next request lands on a known-good token.
What people actually say about Proxy — is it worth it?
We ran a structured research pass across product reviews, community discussions, and post-purchase forum threads to surface the patterns vendors won't publish themselves. Below: the recurring strengths, the hidden costs people mention most, and the cohort that consistently regrets adopting this tool.
105 mentions across 6 sources (Hacker News, Product Hunt, App Store, Bluesky, Stack Overflow, Lemmy) · researched Jul 6, 2026.
Average across the 6 sources that answered — each source counts once, not each post.
- +Open-source MIT license allows full code transparency and customization.
- +Local proxy adds zero latency overhead (~6ms) and no cloud dependency.
- +Smart model routing cuts API costs by assigning cheaper models to simple tasks.
- +Multi-credential pool avoids rate limits by round-robining across keys.
- +Anomaly detection catches token explosions and repetition loops automatically.
- −No community feedback available to verify claimed features or reliability.
- −Telemetry on by default may conflict with local-first privacy expectations.
- −Response cache is exact-match only, limiting benefit for varied prompts.
- −Dashboard requires localhost access; no cloud monitoring option.
- −Potential credential leakage if proxy is exposed on a network.
- • Self-hosting infrastructure (server, networking, maintenance)
- • Potential API costs if using paid model keys through the proxy
Viability Score
How well maintained and how widely used is Proxy? Built from what the vendor actually publishes (docs, changelog, tutorials, integrations, pricing), whether the site is live, and how much real users discuss it. How we calculate this
Last calculated: October 2026
How we score →Key Features
- Local LLM proxy on localhost:4100 with per-request metering
- Routing by task class: simple, moderate, complex, elite
- Per-agent model pinning via JSON config edit, no code changes
- Per-run cost attribution via relayplane run --label across agents, threads, and retries
- Per-agent cost tracking via system-prompt fingerprinting, no annotation
- Local dashboard at localhost:4100 with per-model and per-agent spend
- relayplane watch terminal cost ticker backed by local /v1/stats/live
- Sliding-window anomaly detection for velocity spikes and repetition loops
- Token-explosion guard that blocks the next call after a runaway thinking loop
- Rolling daily, hourly, and per-request budget caps with 429 budget_exceeded
- Configurable breach actions: block, downgrade, warn, alert
- Kill switch: relayplane kill and relayplane resume
- Credential pool with round-robin rotation and quarantine after 2 consecutive 401s
- Quota-aware failover and circuit breaker before 429s cascade
- Cache-aware accounting for Anthropic prompt caching; exportable SQLite request ledger
About Proxy
RelayPlane is a free, MIT-licensed LLM proxy that runs on your own machine and sits between your AI agents and their model providers. You install it with one npm command, point ANTHROPIC_BASE_URL or OPENAI_BASE_URL at localhost:4100, and every request from Claude Code, Cursor, OpenClaw, or any agent that speaks those APIs is logged. The version live on npm is v1.9.70, and the vendor says it has been routing its own autonomous engineering pipeline through the proxy for about four months. The mechanism that cuts the bill is routing by task class. Simple work like scoring, dedup, classification, and voice checks goes to a Haiku-, Groq Llama-, or GPT-4o-mini-class model at roughly $0.10 to $0.80 per million tokens, while code review and judgment work keeps a frontier model. The vendor's own pipeline reports 58% simple / 24% moderate / 13% complex / 5% elite traffic, and its stated claim is that most agent pipelines are 60 to 80% routine. You change models by editing a JSON role entry, not application code — the vendor's example flips a coder role to Kimi K3 on OpenRouter and back. What's finished on day one is the observation layer: a SQLite request ledger on your disk, a dashboard at localhost:4100, a `relayplane watch` terminal cost ticker backed by a local /v1/stats/live endpoint, and `relayplane run --label nightly-backfill -- ./orchestrate.sh` to wrap any command in a per-run rollup split by agent, retries, and model drift. Cost attribution per agent comes from system-prompt fingerprinting, so no annotation work. Anthropic prompt caching is accounted for, and the ledger can be exported. Guards that ship today include sliding-window detection for velocity spikes, cost acceleration, repetition loops, and token explosions (the vendor cites a 72,900-token thinking loop at $40/hour), plus per-agent daily, hourly, and per-request caps with block, downgrade, warn, or alert actions. A kill switch (`relayplane kill` / `relayplane resume`) is live; a credential pool
Behind the Verdict
We'd reach for RelayPlane the moment an agent pipeline starts spending real money and nobody can say which step spent it. The per-run rollup is the part that earns its keep: put `relayplane run --label` in front of a job and you get cost split by agent, retries counted separately from work, and expected bands that flag when today is not like the others. Nothing is instrumented inside your agent, which is why it works with Cursor and OpenClaw as readily as a homegrown loop. The case for it weakens if you need a shared view. A local proxy cannot merge a run spanning CI and three laptops, and the vendor says so on its own pricing page — the hosted Team ($49/mo) and Scale ($199/mo) cards exist to test demand and are not built. If cross-machine history is a hard requirement today, you are shopping for a different product, not waiting on this one. Expectation to set: RelayPlane is an observation and circuit-breaker layer, not an evaluator. The Verify pillar — scoring a diff against acceptance criteria before an agent marks a task done — runs as a separate evaluator in the vendor's pipeline and is on the roadmap, not in the box. Govern is only partly done too; budget caps, anomaly detection, the credential pool, and quota-aware failover are live, while the kill-switch HTTP endpoint is listed as next. Against LiteLLM, the split is governance placement and setup effort. LiteLLM is a self-hosted gateway you configure as infrastructure; RelayPlane is a single-machine proxy you drop in front of an agent with an env var, and its routing is by task class with no code change. Against Helicone, RelayPlane keeps the ledger local and the spend caps enforce at the proxy rather than in a hosted dashboard — the tradeoff is you lose the team-shared UI until the hosted layer
Researching Proxy? Get your full AI stack in 60 seconds.
Free, no signup — tell us your goal and get tools matched to your budget & existing stack.
Real-world workflow fit
Concrete scenarios for the personas Proxy actually fits — and what changes day-one when you adopt it.
Installs with `npm install -g @relayplane/proxy`, points ANTHROPIC_BASE_URL at localhost:4100, and wraps the job in `relayplane run --label nightly-backfill -- ./orchestrate.sh`.
Outcome: Gets a per-run rollup showing cost split by agent with retries tracked separately — the vendor's own sample shows $6.41 notional against a $19.02 all-Opus baseline — plus a hard daily cap that blocks the run if something loops.
Each developer runs the local proxy and routes routine classification calls to a cheap model tier while keeping code review on a frontier model, editing role-to-model mappings in a single JSON config.
Outcome: Individual cost visibility per machine is immediate; merging those runs into one shared dashboard is not, because the $49/mo Team tier that would do it is still a waitlist.
Runs RelayPlane fully locally with telemetry disabled via `relayplane telemetry off`, keeping the SQLite ledger and dashboard on their own infrastructure.
Outcome: Full per-agent and per-model cost tracking with nothing uploaded and no login, and a kill switch via `relayplane kill` if a runaway is detected mid-run.
Use Cases
- Route classification, dedup, and scoring calls to a Haiku- or GPT-4o-mini-class model instead of paying frontier rates for routine work
- Cap a single runaway agent job with a per-run budget ceiling without stopping the whole machine
- Wrap a nightly orchestration script in `relayplane run --label` to see which agent step actually spent the money
- Pool multiple Anthropic subscription credentials so a run doesn't die when one hits its quota wall
- Catch an agent stuck in a repetition or thinking loop before it burns tens of thousands of tokens over 21 minutes
- Keep prompt and response content on your own disk instead of routing it through a hosted observability vendor
- Track retries as a share of run spend to see how much of a job was the same request twice
- Flip a role from Claude to a cheaper OpenRouter model by editing two fields in one JSON file and watch the live cost pane
Models Under the Hood
as of 2026-10-17
Limitations
- RelayPlane is a local proxy by default, so the machine hosting it must stay running and reachable by your agents; no telemetry or account is required, but the hosted Team ($49/mo) and Scale ($199/mo) tiers are an explicit early-access demand test the vendor says is not built, so hosted run history, team dashboards, cross-machine run attribution, and Slack/email/Telegram/PagerDuty alert routing are not available today.
- Budget guards and anomaly detection rely on proxy-side token counting and a price table the vendor verifies against provider pricing pages, so ledger figures are list-price estimates rather than the provider's actual invoice.
- The published CLI's cost ticker is backed by a local /v1/stats/live endpoint polled every 3 seconds by default.
as of 2026-10-04
Verification history
We have re-verified Proxy 10 times since . Each pass re-reads the vendor's own pages and re-checks every listed field against that evidence; passes where nothing had changed are marked as such.
- — re-verified summary, description, our verdict, our analysis, pricing model, pricing tiers, features, integrations, who it suits, who should skip it
- — re-verified summary, description, our verdict, our analysis, pricing model, pricing tiers, features, integrations, who it suits, who should skip it
- — re-verified summary, description, our verdict, our analysis, pricing model, pricing tiers, features, integrations, who it suits, who should skip it
- — re-verified summary, description, our verdict, our analysis, pricing model, pricing tiers, features, integrations, who it suits, who should skip it
- — re-verified summary, description, our verdict, our analysis, pricing model, pricing tiers, features, integrations, who it suits, who should skip it
- — re-verified summary, description, our verdict, our analysis, pricing model, pricing tiers, features, integrations, who it suits, who should skip it
Showing the 6 most recent of 10 verification passes.
Free to cite with attribution — this page re-verifies continuously.
12-month cost
Project the real annual outlay, including the implied monthly cost when only an annual tier is published.
Vendor list price only. Add-on usage, seat overages, and contract minimums are surfaced under Hidden costs & gotchas.
Plans compared
For each published Proxy tier: who it actually fits, and what it adds vs. the previous tier. Cross-reference the cost calculator above for projected annual outlay.
Open Source
$0
Ideal for
Individual developers and privacy-sensitive shops running coding agents locally who want full metering and guards without an account or a card.
What this tier adds
Starting tier: free forever under MIT, with the full proxy, local dashboard, ledger, budget caps, and kill switch included — no feature is gated behind a paid tier.
Team (early access waitlist — not built)
$49/mo
Scale (early access waitlist — not built)
$199/mo
Where the pricing makes sense
The company stage and team size where Proxy's pricing actually pencils out — and where peers do it cheaper.
The local product is free forever under MIT, which undercuts every hosted gateway in the category on price and every managed alternative on data residency. The published Team tier is $49/mo and Scale is $199/mo, both early-access waitlist only with no card required. Against Helicone, Langfuse, or LiteLLM Cloud, RelayPlane's honest pitch is that you pay nothing for the local layer and only consider $49/mo if and when the cross-machine team features actually ship.
Setup time & first value
How long it actually takes to get something useful out of Proxy — broken out by persona, not the marketing-page minute.
Solo developer on a laptop: about 30 seconds to install the npm package and start the proxy, another minute to export ANTHROPIC_BASE_URL or OPENAI_BASE_URL and confirm traffic lands in the dashboard. Small team: under 5 minutes per machine plus whatever time you spend writing the JSON routing config for your agent roles. Teams wanting cross-machine rollups should budget indefinite wait time,
Switching to or from Proxy
How to bring data in from common predecessors and how to get it back out — written for the switcher, not the buyer.
- →From LiteLLM: keep your provider keys, point the same base URL at localhost:4100, and use per-agent model pinning in JSON instead of LiteLLM's config YAML
- →From Helicone or Langfuse hosted observability: disable telemetry-uploading SDKs, route traffic through the local proxy, and read attribution from the SQLite ledger instead of a hosted dashboard
- →From direct provider API calls: set ANTHROPIC_BASE_URL or OPENAI_BASE_URL to http://localhost:4100 and drop the proxy in front with no agent code changes
- →From OpenRouter-only routing: add native Anthropic, OpenAI, Google, xAI, and Ollama credentials alongside OpenRouter so the proxy can fail over before a 429 cascades
- ↗To LiteLLM: lift your JSON role mappings into LiteLLM's config format and accept that per-run cost attribution and runaway-loop guards are not part of the move
- ↗To Helicone or Langfuse: re-instrument your agents with the vendor SDK or OTel spans, since the proxy's transparent request interception does not carry over
- ↗To OpenRouter as a managed router: point your base URL at OpenRouter directly and lose the local ledger, budget caps, and credential pool
Integrations
Resources & Guides
- Documentationrelayplane.com
Docs · Proxy
Full product docs from relayplane.com
- Quickstartrelayplane.com
Quickstart · Proxy
Get up and running fast from relayplane.com
- Documentationrelayplane.com
Installation · Proxy
Full product docs from relayplane.com
- Documentationrelayplane.com
Openclaw Setup · Proxy
Full product docs from relayplane.com
- Documentationrelayplane.com
Cost Optimization · Proxy
Full product docs from relayplane.com
Tutorials & Learning
YouTube returned 6 videos for “Proxy”, and we withheld 6: 6 could not be judged, because “Proxy” is a single word that other videos use for other things. We are showing none, because we could not prove any of them are about Proxy.
Official links
Featured Head-to-Head Comparisons
Proxy vs Spider Cloud
If you're running AI coding agents like Claude Code and want to slash API costs while keeping full data control, Proxy is the clear choice — it's free, open-source, and actively adding cost-tracking features. If you need real-time web data for RAG or agent workflows, Spider Cloud's high-speed Rust crawling and AI extraction (including new Browser AI commands) are more relevant. The two tools solve different problems: Proxy optimizes LLM spend, Spider Cloud feeds agents with fresh external content.
Proxy vs Temporal Ai
Temporal AI and Proxy serve completely different needs: Temporal is for building resilient, long-running workflows that survive failures, while Proxy slashes LLM API costs via smart routing and budget caps. Choose Temporal if you need durable AI agent pipelines; choose Proxy if your primary pain point is runaway LLM costs.
Proxy vs Presto Voice
Presto Voice and Proxy are incomparable in purpose and audience. Presto Voice is a specialized enterprise voice AI for QSR drive-thrus, while Proxy is an open-source cost-saving tool for LLM developers. Your choice depends on whether you run a chain of fast-food restaurants or manage AI agent costs.
Popular in LLM Gateways & Model Routers
OpenRouter Agents
Route text, image, video, and audio models through one OpenAI-compatible API — 500+ models, 80+ providers, no subscriptions.
Intrascope
Intrascope is an AI control layer: multi-model chat with governance, budgets and shared company context.
Frequently Asked Questions
Categories
Best-of guides
Topics
Used Proxy? Help shape our editorial sentiment research.