Alternatives to Evidently AI

OpenJudge is an Apache 2.0 open-source AI evaluation framework shipping 50+ benchmark-validated graders for agents, LLMs, code, math, and multimodal models.

Open-source Python framework that replaces vibe checks with reproducible evaluation loops for RAG pipelines and AI agents.

Open-source LLM observability and agent evaluation you can trace, evaluate, and self-host for free.

Open-source OpenTelemetry tracing, versioning, and sessions for Python LLM apps—bring your own backend.

Open source AI engineering platform for agent and LLM tracing, evaluation, prompt management, and ML lifecycle work.

OpenCompass (司南) benchmarks LLMs, VLMs, and AI4S models on 100+ open evaluation datasets with dated leaderboards

Arize Phoenix is open-source LLM observability: trace agent steps, run evals, and self-host your traces.

Silico is Goodfire's interpretability agent for understanding and debugging the internals of your AI models.

Open-source LLM observability and agent evals: trace, evaluate, and improve production AI agents with MIT-licensed tooling.

Athina AI is a collaborative AI development platform for managing prompts, running LLM evaluations, and monitoring production traces in one

Open-source AI coworker workspace that builds, schedules and ships agents to Slack or Telegram

Maxim AI evaluates, simulates and observes AI agents — with Bifrost, its open-source LLM gateway, handling routing, budgets and guardrails.

Open-source AI observability for agent tracing, LLM-as-a-judge evals, and coding-agent cost tracking.

Open-source agent observability that auto-captures every LLM and tool call and flags failures you never wrote a detector for.

Free open-source AI usage tracking and analytics that syncs your token and cost data from Claude, Cursor, ChatGPT and other tools onto a shareable dashboard.

OpenLIT is an open-source, OpenTelemetry-native agent harness engineering platform for LLM tracing, evals, and guardrails you self-host.

Dash0 is OpenTelemetry-native observability where Agent0 investigates incidents, finds root cause, and opens fix PRs.

Galileo AI Evals turns offline evaluations into production guardrails that gate your AI agents in real time.

LLM evaluation, observability, and red teaming in one workspace — ship AI you can actually trust

TruLens traces AI agent runs with OpenTelemetry-native evals, showing where an agent fails and where you can cut cost.

ToolSpend tracks AI API spend across OpenAI, Anthropic, Google AI, Bedrock and Azure in one live cost dashboard.

Weights & Biases tracks ML experiments and traces LLM apps so teams ship AI models faster

Neptune.ai is an experiment tracker for real-time training monitoring, run comparison, and layer-level metric analysis at frontier scale.

Pre-trained finance AI models and API infrastructure for banks, insurers, and lenders, moving into agentic workflow orchestration.

TruEra is an ML monitoring and AI quality platform that has agreed to join Snowflake; its homepage redirects there.

Developer observability platform that traces, replays, and debugs AI agent runs across OpenAI, CrewAI, Autogen and 400+ LLMs

NannyML estimates your ML model's live performance without ground-truth labels and alerts you only when it actually matters

Free daily LLM speed benchmarks — compare time-to-first-token and tokens-per-second across models and regions.

Replay your production LLM traffic on candidate models to see which behaviours change before you switch.

Metoro is a Kubernetes observability and AI SRE agent that watches your cluster with eBPF and opens fix PRs for production issues.