Arize Phoenix
Arize Phoenix is open-source LLM observability: trace agent steps, run evals, and self-host your traces.
Phoenix remains the pick when traces must stay inside your own infrastructure and you want evals in the same tool. Session-level annotation queues, dataset-driven experiments, OTel-native spans, and ELv2 licensing at zero cost are hard to argue with. Budget for the split: the free AX tier caps at 10 Signal issues and 25k spans/mo, and Pro runs $50/mo once you outgrow it.
Verified 3h ago · liveness 89/100 · cite: rightaichoice.com/tools/arize-phoenix
- AI engineers debugging multi-step agent failures that plain logs can't explain
- Teams building eval suites with LLM-as-judge, Decision Judge, and human annotation queues
- Privacy-sensitive orgs that need traces to stay on their own infrastructure
- Platform teams already running OpenTelemetry who want one tracing pipeline
- Non-technical users who want a managed dashboard with no instrumentation work
- Teams that outgrow 10 Signal issues and 25k spans/mo but won't pay the $50/mo Pro tier
- Buyers who want fully automated quality scoring without defining judges or reviewing annotations
We scan live Reddit threads, YouTube comments, X posts, G2 reviews and other communities — and hand you an honest verdict in under a minute.
- Honest verdict, not marketing
- Real pros & cons from real users
- Attributed quotes with receipts
3 free scans · no card needed
Skip Phoenix if you want a fully managed dashboard with zero instrumentation work, or if you need Signal's automated failure detection and managed agents without paying for Arize AX.
Signal is an Arize AX feature with monthly caps — 10 issues/mo on AX Free, 25 on AX Pro — so failure detection that outpaces your limit pushes you up a tier.
Phoenix's OSS core is free under ELv2 and self-hosted, so the price you pay is your own infrastructure and the engineering time to write evaluators. The managed side is Arize AX: AX Free at $0 with 25k spans/mo and 15-day retention, AX Pro at $50/mo with 50k spans and 30-day retention, and AX Enterprise custom-priced (startup pricing available). Against hosted-only peers, AX Pro's $50/mo is mid-market; the real cost is that Signal, Alyx, and managed agents sit above the free tier.
In short
Arize Phoenix — Arize Phoenix is open-source LLM observability: trace agent steps, run evals, and self-host your traces. Best for AI engineers debugging multi-step agent failures that plain logs can't explain, Teams building eval suites with LLM-as-judge, Decision Judge, and human annotation queues, Privacy-sensitive orgs that need traces to stay on their own infrastructure. Free to start; paid plans from $50/mo.
What's new in Arize Phoenix
Checked yesterdayAcross the latest 2 updates: 2 changelog entries.
Jev-as-a-Judge renamed Decision Judge, now provider-neutral
Jev-as-a-Judge is now Decision Judge, a provider-neutral evaluator type. Runs shared state and typed boolean, choice, and score questions on TypeSafe AI jev-latest or OpenAI gpt-6-luna via existing OpenAI integration. Existing evaluators keep working unchanged.
Webhooks can now be created and managed via REST API, SDKs, and CLI
Webhooks can be created and subscribed to prompt and evaluator events from the REST API, Python/TypeScript/Go SDKs, or the ax CLI. A subscription links one webhook to one event on one prompt or evaluator. Endpoints are in alpha.
What people actually say about Arize Phoenix — is it worth it?
We ran a structured research pass across product reviews, community discussions, and post-purchase forum threads to surface the patterns vendors won't publish themselves. Below: the recurring strengths, the hidden costs people mention most, and the cohort that consistently regrets adopting this tool.
12 mentions across 3 sources (Hacker News, YouTube, Lemmy) · researched Oct 14, 2026.
Weighted by the 27 posts each of 3 sources contributed.
- +OpenTelemetry-native tracing means spans land in existing pipelines with no proprietary backend lock-in.
- +No model, framework, or language lock-in — integrations span LlamaIndex, LangChain, OpenAI, and beyond.
- +Five-step observe-annotate-hypothesize-experiment-measure loop gives evals a structured workflow, not just a dashboard.
- +Self-hosting via uvx, Docker, or Helm on Kubernetes keeps sensitive traces inside your own infrastructure.
- +Session evals score full multi-turn conversations, not just isolated spans — rare among OSS competitors.
- −Requires manual instrumentation and quality criteria before eval scores carry any meaning.
- −Autonomous features like Alyx and dashboard-building agents sit on the paid AX side, not OSS.
- −Getting Claude Code and other tools to emit spans into Phoenix still takes proxy or custom work.
- −Community discussion is concentrated in HN and tutorial videos — thinner elsewhere than Langfuse or LangSmith.
- −Skeptics note a thin trace logger plus a notebook may cover much of the same ground for small teams.
- • Self-hosting means paying for the Helm/Kubernetes infrastructure, storage, and operational headcount despite the $0 license
- • The most-hyped autonomous features (Alyx, PXI, Evaluator Hub) sit behind paid AX tiers, so 'free' only covers the tracing and eval core
- • No published list pricing for Phoenix Cloud or AX — budgeting requires a sales conversation
Viability Score
How well maintained and how widely used is Arize Phoenix? Built from what the vendor actually publishes (docs, changelog, tutorials, integrations, pricing), whether the site is live, and how much real users discuss it. How we calculate this
Last calculated: October 2026
How we score →Key Features
- End-to-end tracing of agent prompts, retrievals, tool calls, and outputs
- OpenTelemetry-native instrumentation with no proprietary lock-in
- LLM-as-judge evaluations on traces, spans, datasets, and experiments
- Decision Judge: provider-neutral typed boolean, choice, or score evaluators
- Session evals: score multi-turn conversations, not just single spans
- Human annotations and labeling queues with review workflows
- Queue full multi-turn sessions and complete traces for annotation
- Annotation queues via app, REST API, and Python, TypeScript, and Go SDKs
- Build datasets directly from production traces
- Run experiments to benchmark changes under identical conditions
- Prompt IDE with multi-prompt comparison and versioning
- PXI: AI engineering agent that investigates, annotates, and experiments via chat
- Alyx with long-term memory, plus dashboard building and widget maintenance
- Multi-modal tracing and evaluation for image, voice, and PDF data
- Agent trajectory visualizations as path and graph views
About Arize Phoenix
Phoenix is the open-source observability and evaluation layer for teams shipping LLM agents — the OSS counterpart to Arize's managed AX platform. Point it at your app and it records every step an agent takes: prompts, retrievals, tool calls, outputs. That telemetry feeds a five-stage loop the docs call observe, annotate, hypothesize, experiment, measure: trace a run, label what broke, build a dataset from those traces, change one thing, rerun under the same conditions, then score cost, latency, and quality. It's built for engineers, not dashboard tourists. Tracing is native OpenTelemetry, so spans land in the pipeline you already run rather than a proprietary backend, and the project claims no model, framework, or language lock-in. Multi-modal tracing covers image, voice, and PDF data. Annotation queues accept full multi-turn sessions and complete traces, not just single spans, driven from the app, the REST API, or the Python, TypeScript, and Go SDKs. Setup is short: `uvx arize-phoenix serve` on your laptop, Docker, Helm on Kubernetes, or managed Phoenix Cloud. The new Agent Native install (`npx skills add Arize-ai/phoenix`) lets coding agents wire up instrumentation for you. Phoenix Cloud pairs with PXI, an AI engineering agent that chats with your traces to investigate issues, add annotations, and run experiments. The paid AX side carries the hosted extras: Alyx retains context across conversations and can build and maintain AX dashboards. Evaluators now include Decision Judge (the renamed Jev-as-a-Judge), a provider-neutral evaluator running typed boolean, choice, or score questions on TypeSafe AI jev-latest or OpenAI gpt-6-luna. Webhooks can be created and subscribed to prompt and evaluator events from the REST API, SDKs, or ax CLI. Versus Langfuse (self-hosted OSS evals) and LangSmith (rented hosted stack), Phoenix assumes an engineer willing to instrument an app and define what a good output looks like before any score means anything.
Behind the Verdict
Reach for Phoenix when the constraint is data residency or a stubborn agent bug you can't reproduce from logs. Self-hosting under ELv2 keeps sensitive traces on your own metal, and OpenTelemetry-native spans mean you aren't rebuilding the pipeline you already run. The observe-to-measure loop is the real product: traces become datasets, datasets become experiments, experiments become scores. Pass if your team wants quality scoring handed to you. Phoenix doesn't guess what 'good' means. Someone has to write evaluators or queue annotations, and until that happens the scores are noise. Non-technical stakeholders eyeing a plug-in dashboard will find the instrumentation work steep. Against Langfuse, the closest OSS alternative, Phoenix leans harder on human annotation workflows and the trace-to-experiment loop; Langfuse is a fine self-hosted choice too, so pick on which eval workflow matches your team rather than raw feature counts. LangSmith is the answer if you'd rather rent a hosted stack and skip the ops. Current eval tooling moved this quarter: Decision Judge is now provider-neutral, so the same typed evaluator runs on TypeSafe AI jev-latest or OpenAI gpt-6-luna through your existing OpenAI integration. Webhooks via REST, SDKs, and the ax CLI make prompt and evaluator events usable for automation. In practice, wire webhooks early so a regression trips an alert instead of surfacing in a support ticket. The split between OSS Phoenix and paid AX is where budgets get caught. The free AX tier's 10 Signal issues and 25k spans per month disappear fast once a real agent runs in production; Pro is $50/mo for 25 issues and 50k spans. Self-hosting Phoenix OSS sidesteps those caps if you're willing to run the infrastructure.
Researching Arize Phoenix? Get your full AI stack in 60 seconds.
Free, no signup — tell us your goal and get tools matched to your budget & existing stack.
Real-world workflow fit
Concrete scenarios for the personas Arize Phoenix actually fits — and what changes day-one when you adopt it.
Run `uvx arize-phoenix serve` locally, add OpenTelemetry tracing to the agent, and watch each prompt, retrieval, and tool call land in Phoenix. Find the span where retrieval returned the wrong document.
Outcome: A specific failing step instead of a vague 'the agent is bad' report, with the trace to show the team.
Build a dataset from production traces, write LLM-as-judge evaluators for the failure patterns, then run an experiment on a changed prompt under the same conditions.
Outcome: A scored before/after on cost, latency, and quality that justifies shipping the change.
Deploy Phoenix with Helm so traces never leave the environment, while keeping OpenTelemetry as the single instrumentation pipeline shared with existing observability tooling.
Outcome: Agent observability and evals within compliance boundaries, with no proprietary lock-in on the trace format.
Use Cases
- Trace every LLM call in your LangChain app to debug latency and errors.
- Evaluate response quality and safety before deploying to production.
- Build a dataset from production traces and benchmark a prompt change against it.
- Send full multi-turn agent sessions to an annotation queue for human review.
- Compare prompts and model outputs side by side in the Prompt IDE.
- Keep sensitive prompts on your own infrastructure by self-hosting Phoenix.
- Export trace data through OpenTelemetry for offline analysis and custom reporting.
Limitations
- Phoenix is the open-source half of a two-product line: the managed Arize AX platform carries Signal, Alyx, managed agents, and self-hosted enterprise deployment.
- Phoenix itself is installed with `uvx arize-phoenix serve` (local) or via Docker/Helm, and the site's tracing setup documents an `npx install` path through a coding agent.
- Evaluator quality depends on what you write — LLM-as-judge, agent-as-a-judge, or code evaluators — and Arize recommends aligning automated scores against human judgment, which is ongoing work.
- Fully automated quality scoring with no human review in the loop is not the model here.
as of 2026-09-29
Verification history
We have re-verified Arize Phoenix 111 times since . Each pass re-reads the vendor's own pages and re-checks every listed field against that evidence; passes where nothing had changed are marked as such.
- — re-checked, vendor evidence unchanged
- — re-verified summary, description, our verdict, our analysis, pricing model, pricing tiers, features, integrations, who it suits, who should skip it
- — re-checked, vendor evidence unchanged
- — re-checked, vendor evidence unchanged
- — re-checked, vendor evidence unchanged
- — re-checked, vendor evidence unchanged
Showing the 6 most recent of 111 verification passes.
Free to cite with attribution — this page re-verifies continuously.
12-month cost
Project the real annual outlay, including the implied monthly cost when only an annual tier is published.
Vendor list price only. Add-on usage, seat overages, and contract minimums are surfaced under Hidden costs & gotchas.
Plans compared
For each published Arize Phoenix tier: who it actually fits, and what it adds vs. the previous tier. Cross-reference the cost calculator above for projected annual outlay.
AX Free
$0/mo
Ideal for
Solo builder or small team evaluating agents with no budget, running 25k spans and 1 GB ingestion a month.
What this tier adds
Free entry point: Signal at 10 issues/mo, 15-day retention, SaaS only, unlimited users and evals.
AX Pro
$50/mo
Ideal for
AI-native team past hobby volume that wants a supported managed instance and email troubleshooting help.
What this tier adds
Doubles spans to 50k/mo and ingestion to 10 GB, extends retention to 30 days, and raises Signal to 25 issues/mo.
AX Enterprise
Custom
Ideal for
Orgs with scaled AI use cases that need SSO, audit logs, RBAC, compliance attestations, or self-hosted deployment.
What this tier adds
Unlimited Signal, custom spans/ingestion/retention, SaaS or self-hosted deployment, custom SLAs, and bring-your-own-key AI providers.
Where the pricing makes sense
The company stage and team size where Arize Phoenix's pricing actually pencils out — and where peers do it cheaper.
Phoenix's OSS core is free under ELv2 and self-hosted, so the price you pay is your own infrastructure and the engineering time to write evaluators. The managed side is Arize AX: AX Free at $0 with 25k spans/mo and 15-day retention, AX Pro at $50/mo with 50k spans and 30-day retention, and AX Enterprise custom-priced (startup pricing available). Against hosted-only peers, AX Pro's $50/mo is mid-market; the real cost is that Signal, Alyx, and managed agents sit above the free tier.
Setup time & first value
How long it actually takes to get something useful out of Arize Phoenix — broken out by persona, not the marketing-page minute.
Local self-host is the fastest path: `uvx arize-phoenix serve` gets a Phoenix instance running in under a minute on your laptop, and the site's tracing install runs through a coding agent. Docker and Helm take longer if you are standing up shared infrastructure. First useful signal arrives once your evaluators and datasets exist, which is engineering work, not a click.
Switching to or from Arize Phoenix
How to bring data in from common predecessors and how to get it back out — written for the switcher, not the buyer.
- →From LangSmith: point your existing OpenTelemetry instrumentation at a self-hosted Phoenix instance and rebuild evaluators as LLM-as-judge or code evaluators.
- →From Langfuse: export traces via the OTel pipeline Phoenix expects and recreate datasets and experiments inside Phoenix.
- →From plain logging: add OpenTelemetry instrumentation to your agent so prompts, retrievals, and tool calls become spans instead of log lines.
- →From a vendor SDK trace format: swap to OTel-native instrumentation so traces are portable rather than pinned to one backend.
- ↗To Arize AX: keep OTel instrumentation and point it at the managed platform for Signal, Alyx, and managed agents.
- ↗To LangSmith: re-instrument with the LangSmith SDK since Phoenix traces are OTel-native rather than LangSmith format.
- ↗To Langfuse: forward OTel spans into Langfuse and rebuild evals and datasets there.
- ↗To a generic OTel backend: keep sending traces to any OTLP-compatible store, since Phoenix does not own the trace format.
Integrations
Resources & Guides
Tutorials & Learning
AIエージェントのための適切な評価（Eval）の構築方法 | Arize Phoenix
Arize AI
Phoenix: Freely Monitor your AI Application Locally
Mervin Praison
Arize Phoenixによるトレースとインストルメンテーションの理解
Data Science Dojo
YouTube returned 6 videos for “Arize Phoenix”, and we withheld 3: 3 did not mention Arize Phoenix. Showing the 3 we can prove are about Arize Phoenix.
Official links
Tools that pair well with Arize Phoenix
Common stack mates teams adopt alongside Arize Phoenix, with the specific reason each pairing earns its keep.
Phoenix
Open-source LLM observability and agent evaluation you can trace, evaluate, and self-host for free.
Langfuse
Open-source LLM observability and agent evals: trace, evaluate, and improve production AI agents with MIT-licensed tooling.
OpenLIT
OpenLIT is an open-source, OpenTelemetry-native agent harness engineering platform for LLM tracing, evals, and guardrails you self-host.
Featured Head-to-Head Comparisons
Swarmtrace vs Arize Phoenix
If your pain is 'why did my multi-agent system do that yesterday?' and you need frame-by-frame replay of every message and state change, SwarmTrace's time-travel debugging is unmatched. But if you're building production LLM apps and want tracing, quality evals, and experiment tracking in one open-source stack that runs anywhere, Arize Phoenix is the safer, more feature-complete default — especially since it's free.
Arize Phoenix vs Nodedb
NodeDB is for teams consolidating multiple datastores into one multi-model engine, ideal for vector+graph hybrid RAG and offline sync. Arize Phoenix is for teams needing deep observability into LLM agent behavior, with tracing, evaluation, and experiment tracking. Choose NodeDB if your pain is database sprawl; choose Phoenix if your pain is untraceable agent failures.
Arize Phoenix vs Skill Seekers
If you need to turn sprawling docs, repos, or PDFs into structured AI skills or RAG pipelines for any platform, Skill Seekers is the clear open-source choice. If you're debugging complex agent traces and evaluating LLM output quality with LLM-as-judge, Arize Phoenix is purpose-built for that. They complement each other: feed Skill Seekers output into Phoenix for observability.
Arize Phoenix vs Thinklabs Ai
If you're a non-technical buyer researching which AI tool to purchase, ThinkLabs AI is your go-to for structured, unbiased comparisons. If you're an AI engineer debugging LLM agent workflows, Arize Phoenix's open-source tracing and evaluation tools are indispensable. Choose based on your role: researcher or builder.
Agnost Ai vs Arize Phoenix
If you need to catch production edge cases that standard evals miss and want a quick, focused monitoring solution, Agnost AI is your pick. But if you want a comprehensive, open-source, self-hostable platform for tracing, evaluating, and iterating on agent performance, Arize Phoenix is the clear winner.
Aegis Latent Core vs Arize Phoenix
If your priority is enforced compliance and verifiable audit evidence for enterprise LLM traffic, Aegis Latent Core is the safer bet. But for AI engineers actively building and debugging agents, Arize Phoenix is the clear winner — it’s open-source, self-hostable, and packed with tracing and evaluation tools. Pick based on whether you need a governance gate or a development workbench.
Traccia vs Arize Phoenix
For AI platform teams that need multi-vendor orchestration and policy governance, Traccia is the control plane you'll want — but it's not something you'll run without an enterprise sales cycle. If you're an AI engineer debugging agent output and iterating on prompts, Phoenix's free, open-source observability and evaluation loop is immediately actionable — and its acquisition by Dynatrace signals deep enterprise backing. Pick based on your primary pain: controlling agents vs. understanding them.
Alternatives to Arize Phoenix
View all
Frequently Asked Questions
Categories
Used Arize Phoenix? Help shape our editorial sentiment research.