Alternatives to Osmosis

Arena AI ranks chat models and coding agents on live head-to-head human votes and real agent sessions.

Open-source benchmark that scores AI agents on real, multi-step software-company work tasks.

AppWorld benchmarks interactive coding agents on 750 multi-app tasks across 457 APIs.

CodeClash benchmarks goal-oriented software engineering by making AI models build and evolve codebases to win 15-round arena tournaments.

OpenJudge is an Apache 2.0 open-source AI evaluation framework shipping 50+ benchmark-validated graders for agents, LLMs, code, math, and multimodal models.

Agent evaluation platform that simulates voice and chat agents, traces production, and scores hallucinations before your customers hit them

LangSmith is agent and LLM observability from the LangChain team: trace, monitor, evaluate, and deploy AI agents.

Synth is a research workbench and hosted compute for optimizing coding agents with reproducible, evidence-backed evals.

Conan is a native macOS HUD for Claude Code that streams every prompt, tool call, skill and token live as the session runs.

Kiln is a local-first AI workbench for building, evaluating, optimizing, and fine-tuning AI systems on your own machine.

Local, MIT-licensed token usage and cost tracking for AI coding agents — reads the logs your agents already write.

Accordion is a free, MIT-licensed pi extension that shows your coding agent's entire context window as blocks you can fold, unfold, and steer.

AgentNotch streams live Claude Code and Codex token and cost telemetry into your Mac's notch

Open-source toolkit for benchmarking vision-language models across 80+ multimodal tasks, with rankings published on the Open VLM Leaderboard.

OpenCode Bar tracks AI coding quota windows, spend, and burn pace from your macOS menu bar

Spool turns your team's AI agent sessions into a shared, searchable library you can read, publish, and resume.

FFMPerative Outrider turns AI research into review-ready draft pull requests matched to your codebase, then measures whether they helped.

Put a hard question to three frontier models — they answer blind, cross-examine each other, and hand back the one answer that survived, with a verdict.

Open-source AI evaluation and observability for LLMs, RAG apps, AI agents, and ML models.

Neptune.ai is an experiment tracker for real-time deep-learning training monitoring, run comparison and layer-level metric analysis

Free Hugging Face agent leaderboard ranking LLMs on planning, tool use, and multi-step agentic tasks

Weights & Biases tracks ML experiments, versions models and datasets, and traces LLM apps — now inside CoreWeave Forge

Patronus AI builds RL environments and training data so frontier agents learn judgment on the job

Compare 400+ AI models on one LLM Stats Score blending benchmarks, output speed and API pricing.

VisualWebArena is a 910-task benchmark from Carnegie Mellon that scores multimodal browser agents on whether they actually complete visual web tasks.

Monte is a post-training and continual learning layer that turns open-weight models into agents trained on your own work.

Burntop is a FastAPI, React 19 and Postgres full-stack template that keeps coding agents honest with 200+ end-to-end tests.

OpenCompass (司南) benchmarks LLMs, VLMs, and AI4S models on dated, dimension-by-dimension leaderboards

QuickCompare replays your production LLM traffic on candidate models so you see which behaviours change before you switch.

OpenPipe's post-training platform now lives on CoreWeave; only the open-source ART reinforcement-learning trainer remains under the OpenPipe name.