Alternatives to Unitxt

Open-source Python library that turns vibe checks into reproducible evaluation loops for RAG pipelines and AI agents.

OpenAI's December 2025 wet-lab evaluation where GPT-5 improved a real molecular cloning protocol 79-fold

Open-source LLM observability: trace agents, manage prompts, and run evals on one MIT-licensed platform.

Open-source LLM observability and agent evaluation you can trace, self-host, and eval end to end.

Open source AI engineering platform for agent and LLM tracing, evaluation, prompt management, and ML lifecycle work.

Open-source AI evaluation and observability for LLMs, RAG apps, AI agents, and ML models.

Open-source Python framework that traces every AI agent span and scores runs with benchmarked, explainable LLM judges.

Open-source OpenTelemetry tracing, versioning, and sessions for Python LLM apps—bring your own backend.

Weights & Biases tracks ML experiments, versions models and datasets, and traces LLM apps — now inside CoreWeave Forge

Open-source benchmark that scores AI agents on real, multi-step software-company work tasks.

Open-source toolkit for benchmarking vision-language models across 80+ multimodal tasks, with rankings published on the Open VLM Leaderboard.

OpenCompass (司南) benchmarks LLMs, VLMs, and AI4S models on dated, dimension-by-dimension leaderboards

OpenJudge is an Apache 2.0 open-source AI evaluation framework shipping 50+ benchmark-validated graders for agents, LLMs, code, math, and multimodal models.

Maxim AI simulates, evaluates and observes AI agents — with Bifrost, its open-source LLM gateway, for routing and guardrails.

Put a hard question to three frontier models — they answer blind, cross-examine each other, and hand back the one answer that survived, with a verdict.

Fiddler AI is an enterprise control plane for agent observability, inline guardrails, and auditable AI governance.

Athina is a shared workspace where your team writes prompts, runs evals, annotates datasets, and traces LLM calls in production.

Laminar delivers open-source AI agent observability that auto-captures every LLM and tool call and surfaces failures with Signals.

Goodfire's Silico is an AI interpretability agent for reverse-engineering, debugging, and steering model internals.

ViDoRe Benchmark: an open visual document retrieval benchmark and ColPali model suite for enterprise RAG evaluation.

Sharbo builds a skill network for AI agents — encode expert workflows once and let every agent activate them autonomously.

MathArena is a free competition-math LLM leaderboard with Lean-verified proofs and clickable raw model outputs.

PandaProbe is open-source agent observability that traces runs, evaluates outcomes, and turns failures into validated fixes via its Repair

Open-source PyTorch toolkit for feature attribution in sequence generation models

LangWatch is an open-source LLMOps platform for agent testing, LLM evaluation and OpenTelemetry-native LLM observability.

Comet's Opik brings LLM observability and evaluation that turns agent traces into code fixes

AI quality workspace for LLM evaluation, production observability, red teaming, and governance

Synth is a research workbench and hosted compute for optimizing coding agents with reproducible, evidence-backed evals.

Track AI spend across OpenAI, Anthropic, Google AI, Bedrock and Azure in one live cost dashboard.

OpenLIT delivers OpenTelemetry-native LLM tracing, evals, guardrails, and prompt management you self-host free under Apache 2.0.