Alternatives to Comet Opik

LangSmith is agent and LLM observability from the LangChain team: trace, monitor, evaluate, and deploy AI agents.

Open-source LLM observability and agent evaluation you can trace, self-host, and eval end to end.

LangWatch is an open-source LLMOps platform for agent testing, LLM evaluation and OpenTelemetry-native LLM observability.

Open-source AI evaluation and observability for LLMs, RAG apps, AI agents, and ML models.

OpenJudge is an Apache 2.0 open-source AI evaluation framework shipping 50+ benchmark-validated graders for agents, LLMs, code, math, and multimodal models.

Open-source LLM observability: trace agents, manage prompts, and run evals on one MIT-licensed platform.

Open source AI engineering platform for agent and LLM tracing, evaluation, prompt management, and ML lifecycle work.

Laminar delivers open-source AI agent observability that auto-captures every LLM and tool call and surfaces failures with Signals.

Open-source prompt management with built-in LLM observability for cost, latency, and token usage

OpenLIT delivers OpenTelemetry-native LLM tracing, evals, guardrails, and prompt management you self-host free under Apache 2.0.

Open-source Python library that turns vibe checks into reproducible evaluation loops for RAG pipelines and AI agents.

PandaProbe is open-source agent observability that traces runs, evaluates outcomes, and turns failures into validated fixes via its Repair

AI quality workspace for LLM evaluation, production observability, red teaming, and governance

Maxim AI simulates, evaluates and observes AI agents — with Bifrost, its open-source LLM gateway, for routing and guardrails.

Open-source AI agent improvement: trace production runs, detect failures, investigate signals, verify fixes with evals.

Synth is a research workbench and hosted compute for optimizing coding agents with reproducible, evidence-backed evals.

Track AI spend across OpenAI, Anthropic, Google AI, Bedrock and Azure in one live cost dashboard.

Autoblocks AI tests, simulates, and monitors LLM agents with evaluation logic built from your own subject-matter experts.

Open-source Python framework that traces every AI agent span and scores runs with benchmarked, explainable LLM judges.

PromptLayer is the collaboration layer for AI engineering teams — a prompt CMS, eval harness, and agent observability stack in one dashboard.

ViDoRe Benchmark: an open visual document retrieval benchmark and ColPali model suite for enterprise RAG evaluation.

Open-source OpenTelemetry tracing, versioning, and sessions for Python LLM apps—bring your own backend.

Local, MIT-licensed token usage and cost tracking for AI coding agents — reads the logs your agents already write.

Open-source benchmark that scores AI agents on real, multi-step software-company work tasks.

Open-source toolkit for benchmarking vision-language models across 80+ multimodal tasks, with rankings published on the Open VLM Leaderboard.

Open-source React components for rendering AI agent traces, tool calls and reasoning inside your own app.

OpenAI's December 2025 wet-lab evaluation where GPT-5 improved a real molecular cloning protocol 79-fold

BALROG is an open ICLR-published benchmark that scores agentic LLM and VLM reasoning across seven procedurally generated games.

AI cost tracking that turns 50+ provider billing files into one normalized spend dashboard.

Open-source collaboration layer that turns domain expertise into AI agent test suites