Alternatives to Matharena

Open-source LLM & VLM evaluation platform for standardized benchmarking

Silico: mechanistic interpretability platform to understand, debug, and design AI models

Fiddler AI is an enterprise AI control plane for agent observability, guardrails, and governance across the agentic lifecycle.

Weights & Biases tracks ML experiments and traces LLM apps so teams can ship AI models faster

Free public leaderboard ranking LLMs on real-world agentic tasks — planning, tool use, and multi-step execution

Arena AI is a free, community-voted LLM leaderboard ranking chat models, agents, and fullstack code on live head-to-head battles.

Open ICLR-published benchmark measuring agentic LLM and VLM reasoning across seven procedurally generated games, with weekly-updated public leaderboards.

Open-source benchmark toolkit for 220+ vision-language models across 80+ tasks, with a public leaderboard.

PostHog: all-in-one analytics, session replay, feature flags, and data warehouse — with a free tier that covers 97% of teams.

Open-source AI evaluation and observability for LLMs, RAG, agents, and predictive ML models.

Agent observability that traces, replays, and debugs AI agent runs across 400+ LLMs and frameworks

Independent AI leaderboard ranking 300+ models by intelligence, speed, and price.

Free, MIT-licensed desktop widget that shows token usage, spend, and quota limits across 29+ AI coding tools on your own machine.

Open-source benchmark for AI agents on multi-step, real-world software company tasks.

Open visual document retrieval benchmark and model suite for enterprise RAG.

Daily-updated LLM speed benchmarks measured across regions with TTFT, TPS, and total time.

Upload your data, compare 50+ LLMs side by side on quality, cost & speed.

Free, MIT-licensed local dashboard that reads your AI coding agents' log files to show tokens, cost, and traces — no SDK or API key.

Free open-source AI usage tracking & analytics for developers

Open-source benchmark for evaluating multimodal web agents on 910 realistic visual tasks.

Interactive coding agent benchmark simulating 9 apps and 457 APIs to evaluate agent reliability.

Open-source benchmark that tests AI agents on real, live websites with two-stage HTTP interception and LLM-judge scoring.

Open-source LLM observability and evals for building reliable agents

OpenTelemetry-native observability with AI SRE Agent0 for automated production insight.

Open-source tracing, evaluation, and prompt iteration for AI agents — self-host it on your own infrastructure with no per-span bill.

LangSmith is an AI agent observability platform for tracing, monitoring, and evaluating LLM apps and long-running agents.

Open-source LLM observability for tracing, evaluating, and optimizing AI agents.

AI observability and evaluation platform that turns offline evals into production guardrails.

AI observability and evals that turn agent traces into git-committed code fixes

Open-source OpenTelemetry observability for Python LLM apps