Alternatives to Burntop

Open-source AI observability for agent tracing, LLM-as-a-judge evals, and coding-agent cost tracking.

Token Monitor is a free desktop widget that tracks AI coding token usage, cost, and quota limits across 44+ tools and providers.

Open-source LLM observability and agent evaluation you can trace, evaluate, and self-host for free.

ToolSpend tracks AI API spend across OpenAI, Anthropic, Google AI, Bedrock and Azure in one live cost dashboard.

Arize Phoenix is open-source LLM observability: trace agent steps, run evals, and self-host your traces.

Open-source LLM observability and agent evals: trace, evaluate, and improve production AI agents with MIT-licensed tooling.

Open-source OpenTelemetry tracing, versioning, and sessions for Python LLM apps—bring your own backend.

TruLens traces AI agent runs with OpenTelemetry-native evals, showing where an agent fails and where you can cut cost.

Open source AI engineering platform for agent and LLM tracing, evaluation, prompt management, and ML lifecycle work.

Open-source AI coworker workspace that builds, schedules and ships agents to Slack or Telegram

Maxim AI evaluates, simulates and observes AI agents — with Bifrost, its open-source LLM gateway, handling routing, budgets and guardrails.

PostHog is product analytics with session replay, feature flags, error tracking, and a context warehouse for you and your AI agents.

Open-source Python framework that replaces vibe checks with reproducible evaluation loops for RAG pipelines and AI agents.

Evidently AI is an open-source Python framework for LLM evaluation, RAG testing, and ML model monitoring under Apache 2.0.

Helicone is an AI gateway and LLM observability platform for routing, debugging, and cost-tracking AI app traffic.

Open-source prompt management with LLM observability for cost, latency, and token usage

OpenCompass (司南) benchmarks LLMs, VLMs, and AI4S models on 100+ open evaluation datasets with dated leaderboards

Free daily LLM speed benchmarks — compare time-to-first-token and tokens-per-second across models and regions.

Local, MIT-licensed token and cost observability for AI coding agents — reads the logs your agents already write.

Open-source agent observability that auto-captures every LLM and tool call and flags failures you never wrote a detector for.

OpenLIT is an open-source, OpenTelemetry-native agent harness engineering platform for LLM tracing, evals, and guardrails you self-host.

OpenJudge is an Apache 2.0 open-source AI evaluation framework shipping 50+ benchmark-validated graders for agents, LLMs, code, math, and multimodal models.

Dash0 is OpenTelemetry-native observability where Agent0 investigates incidents, finds root cause, and opens fix PRs.

Developer observability platform that traces, replays, and debugs AI agent runs across OpenAI, CrewAI, Autogen and 400+ LLMs

Langfuse Prompt Experiments links LLM tracing, prompt versioning, and evaluators so you can test prompts against real production data.

Free public leaderboard benchmarking LLMs on competition math and Lean proof benchmarks like ArXivMath and ArXivLean.

Metoro is a Kubernetes observability and AI SRE agent that watches your cluster with eBPF and opens fix PRs for production issues.

Chronicle Labs replays your production data to test and validate AI agents before launch.

Silico is Goodfire's interpretability agent for understanding and debugging the internals of your AI models.

LangSmith is agent and LLM observability: trace, monitor, and evaluate AI agents in cloud, BYOC, or self-hosted.