Alternatives to MLflow

Open-source tracing, evaluation, and prompt iteration for AI agents — self-host it on your own infrastructure with no per-span bill.

Open-source LLM observability, prompt management, and evaluation for teams running AI agents in production.

Open-source, OpenTelemetry-native platform for LLM tracing, evaluation, prompt management, guardrails, and GPU monitoring you host yourself.

Open-source, OpenTelemetry-native evaluation and tracing that shows exactly where your AI agent fails — and where you can cut cost.

Open-source Python framework that replaces vibe checks with reproducible evaluation loops for RAG pipelines and AI agents.

Open-source LLM observability, prompt management, and agent evals in one MIT-licensed platform.

OpenJudge is an open-source AI evaluation framework with 50+ production-grade graders for agents, multimodal models, code, and math.

Open-source workspace for building, evaluating, and deploying AI agents you talk to in Slack, Telegram, or a browser chat

Maxim AI simulates, evaluates, and observes AI agents in one workspace, with an open-source gateway (Bifrost) for routing and governance.

Open-source Python framework for evaluating and monitoring LLMs, RAG apps, AI agents, and predictive ML models.

Open-source agent tracing, LLM-as-a-judge evals, and coding-agent cost tracking you can self-host.

Microsoft's open-source, code-first Python framework for building stateful, data-intensive conversational AI agents.

Arize Phoenix is open-source LLM observability: trace every agent step, run evals, and self-host the whole thing.

Opik, Comet's open-source LLM observability and eval platform, turns agent traces into root-cause groupings and git-committed code fixes

Open-source OpenTelemetry tracing, versioning, and sessions for Python LLM apps—bring your own backend.

Collaborative AI development platform for prompt management, dataset evaluation, and production LLM monitoring.

Open-source RAG engine that turns messy documents into a trustworthy context layer for AI agents, with ETL, hybrid search and agentic retrieval.

Open-source agentic data assistant: your LLM connects to databases, writes SQL and code, and runs analysis in sandboxes

Agno is an open-source Python SDK plus AgentOS runtime for building, serving, and operating self-hosted agent platforms.

Open-source agent platform that fuses knowledge graphs with RAG for self-hosted, explainable enterprise AI.

Open-source, OpenTelemetry-native observability for AI agents that catches failures automatically and helps you fix them.

Engineering analytics that attributes every commit, prompt, and token to a human or an AI agent and scores the return on your AI coding spend.

OpenAgents is an Apache-2.0 platform for language agents that analyze data, call 200+ plugins and browse the web.

Discontinued: WhyLabs ceased operations in 2025 and open-sourced its platform; whylogs and langkit live on as open source.

Enterprise agent workforce platform for building, operating, and governing AI agents across on-prem, hybrid, and multi-cloud environments.

Fiddler AI is an enterprise AI control plane for agent observability, guardrails, and governance across the agentic lifecycle.

AI observability and eval engineering platform that turns offline evals into live production guardrails for agents and RAG systems.

Pre-trained finance AI models and API infrastructure for banks, insurers, and lenders, moving into agentic workflow orchestration.

Developer observability platform that traces, replays, and debugs AI agent runs across OpenAI, CrewAI, Autogen and 400+ LLMs

OpenCompass (司南) benchmarks LLMs, VLMs, and AI4S models against 100+ open evaluation datasets with published, dated leaderboards