Alternatives to Token Monitor

Free open-source AI usage tracking and analytics that syncs your token and cost data from Claude, Cursor, ChatGPT and other tools onto a shareable dashboard.

ToolSpend tracks AI API spend across OpenAI, Anthropic, Google AI, Bedrock and Azure in one live cost dashboard.

Free daily LLM speed benchmarks — compare time-to-first-token and tokens-per-second across models and regions.

Local, MIT-licensed token and cost observability for AI coding agents — reads the logs your agents already write.

Open-source AI observability for agent tracing, LLM-as-a-judge evals, and coding-agent cost tracking.

Free MIT-licensed macOS menu bar app that tracks quota windows, spend, and burn pace for 13 AI coding providers

Open-source LLM observability and agent evaluation you can trace, evaluate, and self-host for free.

LangSmith is agent and LLM observability: trace, monitor, and evaluate AI agents in cloud, BYOC, or self-hosted.

TruLens traces AI agent runs with OpenTelemetry-native evals, showing where an agent fails and where you can cut cost.

Athina AI is a collaborative AI development platform for managing prompts, running LLM evaluations, and monitoring production traces in one

Weights & Biases tracks ML experiments and traces LLM apps so teams ship AI models faster

Evidently AI is an open-source Python framework for LLM evaluation, RAG testing, and ML model monitoring under Apache 2.0.

Neptune.ai is an experiment tracker for real-time training monitoring, run comparison, and layer-level metric analysis at frontier scale.

TruEra is an ML monitoring and AI quality platform that has agreed to join Snowflake; its homepage redirects there.

Developer observability platform that traces, replays, and debugs AI agent runs across OpenAI, CrewAI, Autogen and 400+ LLMs

Helicone is an AI gateway and LLM observability platform for routing, debugging, and cost-tracking AI app traffic.

Free public leaderboard benchmarking LLMs on competition math and Lean proof benchmarks like ArXivMath and ArXivLean.

Arize Phoenix is open-source LLM observability: trace agent steps, run evals, and self-host your traces.

Dash0 is OpenTelemetry-native observability where Agent0 investigates incidents, finds root cause, and opens fix PRs.

Silico is Goodfire's interpretability agent for understanding and debugging the internals of your AI models.

Open-source LLM observability and agent evals: trace, evaluate, and improve production AI agents with MIT-licensed tooling.

Galileo AI Evals turns offline evaluations into production guardrails that gate your AI agents in real time.

Opik by Comet turns LLM traces into root-cause groupings, eval scores and code fixes so agent teams stop shipping the same bug twice

Open-source OpenTelemetry tracing, versioning, and sessions for Python LLM apps—bring your own backend.

LLM evaluation, observability, and red teaming in one workspace — ship AI you can actually trust

Open source AI engineering platform for agent and LLM tracing, evaluation, prompt management, and ML lifecycle work.

Fiddler AI is an enterprise AI control plane for agent observability, inline guardrails, and auditable AI governance.

Open-source AI coworker workspace that builds, schedules and ships agents to Slack or Telegram

Maxim AI evaluates, simulates and observes AI agents — with Bifrost, its open-source LLM gateway, handling routing, budgets and guardrails.

Honeycomb Query Assistant turns plain-English questions into HQL against live production telemetry.