Alternatives to NannyML

Open-source AI evaluation and observability for LLMs, RAG apps, AI agents, and ML models.

Helicone is an AI gateway and LLM observability platform that routes, caches, and logs traffic across 100+ model providers.

Put a hard question to three frontier models — they answer blind, cross-examine each other, and hand back the one answer that survived, with a verdict.

Laminar delivers open-source AI agent observability that auto-captures every LLM and tool call and surfaces failures with Signals.

Track AI spend across OpenAI, Anthropic, Google AI, Bedrock and Azure in one live cost dashboard.

Conan is a native macOS HUD for Claude Code that streams every prompt, tool call, skill and token live as the session runs.

Goodfire's Silico is an AI interpretability agent for reverse-engineering, debugging, and steering model internals.

OpenLIT delivers OpenTelemetry-native LLM tracing, evals, guardrails, and prompt management you self-host free under Apache 2.0.

Weights & Biases tracks ML experiments, versions models and datasets, and traces LLM apps — now inside CoreWeave Forge

MathArena is a free competition-math LLM leaderboard with Lean-verified proofs and clickable raw model outputs.

OpenCompass (司南) benchmarks LLMs, VLMs, and AI4S models on dated, dimension-by-dimension leaderboards

QuickCompare replays your production LLM traffic on candidate models so you see which behaviours change before you switch.

OpenJudge is an Apache 2.0 open-source AI evaluation framework shipping 50+ benchmark-validated graders for agents, LLMs, code, math, and multimodal models.

Touchmark is a live order book for buying and selling reserved AI inference capacity at forward-curve prices.

Zero-config local dashboard that shows what your OpenClaw agents actually did, what they cost, and whether they leaked secrets

Open-source LLM observability: trace agents, manage prompts, and run evals on one MIT-licensed platform.

Open-source LLM observability and agent evaluation you can trace, self-host, and eval end to end.

Comet's Opik brings LLM observability and evaluation that turns agent traces into code fixes

AI quality workspace for LLM evaluation, production observability, red teaming, and governance

Open source AI engineering platform for agent and LLM tracing, evaluation, prompt management, and ML lifecycle work.

Maxim AI simulates, evaluates and observes AI agents — with Bifrost, its open-source LLM gateway, for routing and guardrails.

Braintrust traces agent runs, scores them with evals, and surfaces production failure patterns before they ship.

LangSmith is agent and LLM observability from the LangChain team: trace, monitor, evaluate, and deploy AI agents.

Fiddler AI is an enterprise control plane for agent observability, inline guardrails, and auditable AI governance.

Galileo turns offline AI evals into production guardrails for agents and RAG pipelines.

Trace, replay and debug AI agent runs — LLM calls, tool use and multi-agent interactions in one timeline, via a single SDK

SuperPenguin is AI spend intelligence that attributes every AI dollar to a customer, feature, team, or merged pull request.

Athina is a shared workspace where your team writes prompts, runs evals, annotates datasets, and traces LLM calls in production.

Cekura tests, monitors, and self-improves voice AI agents with automated QA loops.

Free daily LLM speed benchmarks — compare time-to-first-token and tokens-per-second across models and regions.