Phoenix
Open-source LLM observability and agent evaluation you can trace, evaluate, and self-host for free.
Phoenix is the first open-source LLM observability tool we'd reach for when debugging a multi-step agent, because it hands you trace-level evidence and a real eval loop without a per-span bill. The catch is operational: Helm upgrades, storage and scaling are on you. If that trade is unwelcome, AX Free's 25k spans and 15-day retention is the natural starting lane, with AX Pro at $50/mo for 50k spans and 30-day retention.
Verified 1d ago · liveness 87/100 · cite: rightaichoice.com/tools/phoenix
- AI engineers debugging multi-step agent workflows with trace-level evidence
- Teams that must keep sensitive prompts and traces inside their own infrastructure
- Developers building eval datasets from real production failures
- Vendor-agnostic stacks already instrumented with OpenTelemetry
- Teams with no appetite for running Docker, Kubernetes, or Helm upgrades
- Prototypes that will exceed AX Free's 25k spans, 1 GB and 15-day retention quickly
- Buyers who want a fully managed SaaS experience with zero infrastructure thinking
We scan live Reddit threads, YouTube comments, X posts, G2 reviews and other communities — and hand you an honest verdict in under a minute.
- Honest verdict, not marketing
- Real pros & cons from real users
- Attributed quotes with receipts
3 free scans · no card needed
Skip Phoenix if nobody on your team wants to own Docker, Kubernetes, or Helm upgrades and you need the managed multi-modal and enterprise features — those live in paid Arize AX tiers, not the free Phoenix OSS build.
AX Free's 25k spans, 1 GB ingestion, and 15-day retention cap out fast in production — exceeding them pushes you to AX Pro at $50/mo.
Phoenix OSS is free to self-host, and AX Free at $0/mo covers small teams with 25k spans and 15-day retention. AX Pro at $50/mo fits AI-native teams needing 50k spans, 10 GB ingestion, and 30-day retention — cheaper than most per-seat managed observability rivals, but you give up their fully managed depth unless you step up to AX Enterprise.
In short
Phoenix — Open-source LLM observability and agent evaluation you can trace, evaluate, and self-host for free. Best for AI engineers debugging multi-step agent workflows with trace-level evidence, Teams that must keep sensitive prompts and traces inside their own infrastructure, Developers building eval datasets from real production failures. Free to start; paid plans from $50/mo.
What's new in Phoenix
Checked 8 days agoAcross the latest 3 updates: 3 feature updates.
Create and manage webhooks from the REST API, SDKs, and CLI
Webhooks can now be created and subscribed to prompt and evaluator events (such as PROMPT_VERSION_CREATED and EVALUATOR_VERSION_CREATED) via REST API, Python, TypeScript, and Go SDKs, or the ax CLI, with bearer-token or HMAC-SHA256 authentication. Endpoints are in alpha.
Alyx can build and maintain dashboards
Arize AX's Alyx agent now creates, edits, and themes dashboards and their widgets, validates that each widget returns data, and fixes empty ones.
Jev-as-a-Judge evaluators in Evaluator Hub
TypeSafe AI's Jev decision model is available as an evaluator, supporting typed boolean, choice, or score questions per record and publishing label and confidence scores.
What people actually say about Phoenix — is it worth it?
We scanned public community sources for Phoenix on Aug 30, 2026 and could not establish that the discussion we found is about this tool rather than something else sharing its name. Our own analysis of that scan says the posts were off-subject. Rather than publish a sentiment score built on the wrong subject, we publish nothing here and re-run the scan.
Viability Score
How well maintained and how widely used is Phoenix? Built from what the vendor actually publishes (docs, changelog, tutorials, integrations, pricing), whether the site is live, and how much real users discuss it. How we calculate this
Last calculated: October 2026
How we score →Key Features
- Trace every agent step: prompts, retrievals, tool calls, and outputs
- LLM-as-judge evaluation for relevance, toxicity, and quality
- Human annotations and labeling queues on traces
- Build datasets from production traces for reproducible experiments
- Benchmark experiments across cost, latency, and performance
- PXI AI engineering agent that investigates issues against traces
- Agent trajectory visualizations with path and graph views
- Session support for multi-turn conversation tracking and session evals
- Native OpenTelemetry support with the OpenInference spec
- Self-host locally with uvx, Docker, or Kubernetes via Helm
- Vendor-agnostic across models, frameworks, and languages
- Online evals on traces and spans; offline evals on datasets
- Alyx builds, edits, and themes dashboards and validates widget data (AX)
- Jev-as-a-Judge TypeSafe evaluators in Evaluator Hub (AX)
- Create and manage webhooks for prompt and evaluator events via REST API, SDKs, or CLI
About Phoenix
Phoenix is Arize AI's open-source observability and evaluation layer for LLM applications and AI agents. It records every step a request takes — prompts, retrievals, tool calls, and final outputs — so a wrong answer becomes a specific failed step in a trace instead of a guess. You run it locally with `uvx arize-phoenix serve`, in Docker, or on Kubernetes via Helm, and traces stay inside your own environment. The built-in workflow follows a loop: observe the trace, annotate what worked and what broke (human review or LLM-as-judge), form a hypothesis, build a dataset from production traces, replay it under identical conditions, then score the result on cost, latency, and quality. PXI, the AI engineering agent, works directly against your traces — investigating issues, adding annotations, and running experiments without leaving the UI. Session support covers multi-turn conversations, and agent trajectory views show both path and graph structure. Instrumentation is standard rather than proprietary: native OpenTelemetry with the OpenInference spec keeps Phoenix vendor-agnostic across models, frameworks, and languages, and coding agents can wire it in via `npx skills add Arize-ai/phoenix`. Phoenix is licensed under ELv2, and the project reports 10k+ GitHub stars, 3M+ monthly downloads and 7k+ community members. Phoenix is best understood as the open-source core of a two-product line. Its sibling Arize AX is the managed platform, and AX's pricing page publishes the deeper tier: AX Free at $0 with 25k spans/month, 1 GB ingestion and 15-day retention, AX Pro at $50/mo with 50k spans, 10 GB and 30-day retention, and AX Enterprise custom with SaaS or self-hosted deployment. Pick Phoenix when you want to own the stack; pick AX when you would rather rent it.
Behind the Verdict
Where Phoenix earns its keep is the debugging loop, not the dashboard. When an agent gives a bad answer, the trace shows which retrieval, tool call, or prompt step actually broke, and from that trace you can build a dataset, replay it, and score the fix. Teams doing this by hand in notebooks will feel the difference immediately. Pick it when two things are true: you already speak OpenTelemetry, and you are willing to run infrastructure. Phoenix's OpenInference spec means traces come out in a format your existing tooling can read, so the switching cost in either direction stays low. Self-hosting also settles the data-residency question — prompts and traces never leave your environment. Pass when nobody on the team wants to own Helm charts, storage sizing, or upgrades. That is not a knock on the product; it is a real cost that shows up around month three. Same for prototypes that expect managed scale out of the box. The closest alternative is Arize AX, Phoenix's managed sibling, and the pricing page makes the split concrete: AX Free gives 25k spans/month with 15-day retention at no cost, while AX Pro at $50/mo raises that to 50k spans, 10 GB ingestion and 30-day retention. Enterprise adds custom volume and either SaaS or self-hosted deployment. If you are choosing between them, the real question is whether observability infrastructure is core to your team or a distraction. Recent AX releases are worth watching because they signal where the platform is heading: Alyx can now build, edit and theme dashboards and validate that each widget returns data; TypeSafe's Jev decision model landed in the Evaluator Hub for typed boolean, choice and score evaluations; and webhooks can be created and subscribed to prompt and evaluator events through the REST API, Python, TypeScript
Researching Phoenix? Get your full AI stack in 60 seconds.
Free, no signup — tell us your goal and get tools matched to your budget & existing stack.
Real-world workflow fit
Concrete scenarios for the personas Phoenix actually fits — and what changes day-one when you adopt it.
Instrument a LangChain agent with OpenTelemetry, run it locally via `uvx arize-phoenix serve`, and replay a production failure to see which tool call or retrieval went wrong.
Outcome: A trace showing the exact failed step, plus an annotation and a dataset row you can replay after the fix.
Stand up Phoenix on Kubernetes via Helm, wire LLM-as-judge evals for relevance and toxicity, and compare two prompt versions against the same dataset.
Outcome: Side-by-side cost, latency, and quality numbers with evidence you can take to a weekly review.
Skip self-hosting and use a free Phoenix Cloud instance or AX Free to trace live traffic and set up human annotations.
Outcome: Trace coverage without running Helm, with a clear upgrade path to AX Pro when spans exceed 25k/month.
Use Cases
- Monitor live LLM responses for hallucinations and bias
- Compare prompt versions to optimize response quality
- Debug latency and token usage in production LLM pipelines
- Wire automated evaluations into CI/CD for AI features
- Trace end-to-end calls across LangChain, LlamaIndex, and custom chains
- Audit LLM outputs for compliance and safety
- Run A/B tests on prompt changes across model providers
- Investigate a failing agent run with PXI instead of reading raw logs
Limitations
- Phoenix is an open-source observability and evaluation platform for AI agents; its managed service (Arize AX) has usage limits.
- AX Free includes 10 Signal issues per month, 25k trace spans per month, 1 GB ingestion volume, 15 days retention, and unlimited users and evals.
- AX Pro costs $50/mo and offers 25 Signal issues, 50k spans, 10 GB ingestion, and 30 days retention; an Enterprise tier offers custom limits.
- Enterprise SSO, audit logs, HIPAA support, and self-hosted AX deployments are not available below the Enterprise tier.
- Multi-modal tracing across image, voice, and PDF is listed as an AX Pro capability.
- The platform is vendor-agnostic and works with any model, framework, or language, so no specific underlying AI models are stated.
- When you self-host Phoenix OSS, upgrades, storage, and Kubernetes operations are your responsibility.
as of 2026-10-10
Verification history
We have re-verified Phoenix 20 times since . Each pass re-reads the vendor's own pages and re-checks every listed field against that evidence; passes where nothing had changed are marked as such.
- — re-verified summary, description, our verdict, our analysis, pricing model, pricing tiers, features, integrations, who it suits, who should skip it
- — re-verified summary, description, our verdict, our analysis, pricing model, pricing tiers, features, integrations, who it suits, who should skip it
- — re-verified summary, description, our verdict, our analysis, pricing model, pricing tiers, features, integrations, who it suits, who should skip it
- — re-verified summary, description, our verdict, our analysis, pricing model, pricing tiers, features, integrations, who it suits, who should skip it
- — re-verified summary, description, our verdict, our analysis, pricing model, pricing tiers, features, integrations, who it suits, who should skip it
- — re-verified summary, description, our verdict, our analysis, pricing model, pricing tiers, features, integrations, who it suits, who should skip it
Showing the 6 most recent of 20 verification passes.
Free to cite with attribution — this page re-verifies continuously.
12-month cost
Project the real annual outlay, including the implied monthly cost when only an annual tier is published.
Vendor list price only. Add-on usage, seat overages, and contract minimums are surfaced under Hidden costs & gotchas.
Plans compared
For each published Phoenix tier: who it actually fits, and what it adds vs. the previous tier. Cross-reference the cost calculator above for projected annual outlay.
Phoenix OSS
$0
AX Free
$0/mo
Ideal for
Small teams or individual builders evaluating Arize's managed platform with light production traffic and short lookback needs.
What this tier adds
Adds SaaS deployment with 25k spans/month, 1 GB ingestion, 15-day retention, 10 Signal issues/month, and unlimited users, evals, experiments, and annotations.
AX Pro
$50/mo
Ideal for
AI-native teams past prototyping that need longer retention, more span headroom, and multi-modal tracing across image, voice, and PDF.
What this tier adds
Lifts limits to 50k spans/month, 10 GB ingestion, 30-day retention, and 25 Signal issues/month for $50/mo, and unlocks custom code evaluators and multi-modal tracing.
AX Enterprise
Custom
Ideal for
Teams with scaled AI use cases that need custom volume, self-hosted deployment, and enterprise controls such as SSO, audit logs, and HIPAA support.
What this tier adds
Removes Signal limits, makes span volume, ingestion, and retention custom, and adds a SaaS-or-self-hosted deployment choice; startup pricing is available.
Where the pricing makes sense
The company stage and team size where Phoenix's pricing actually pencils out — and where peers do it cheaper.
Phoenix OSS is free to self-host, and AX Free at $0/mo covers small teams with 25k spans and 15-day retention. AX Pro at $50/mo fits AI-native teams needing 50k spans, 10 GB ingestion, and 30-day retention — cheaper than most per-seat managed observability rivals, but you give up their fully managed depth unless you step up to AX Enterprise.
Setup time & first value
How long it actually takes to get something useful out of Phoenix — broken out by persona, not the marketing-page minute.
Self-hosting Phoenix OSS is a one-command start (`uvx arize-phoenix serve`), so a first trace can appear in under an hour. Wiring instrumentation across an existing codebase typically takes a day. Helm-based Kubernetes deployments take longer depending on your cluster setup. Two free Phoenix Cloud instances or AX Free get a small team to first value the same day without any infrastructure work.
Switching to or from Phoenix
How to bring data in from common predecessors and how to get it back out — written for the switcher, not the buyer.
- →From raw log-based debugging: instrument with OpenInference, then replace grep-and-read workflows with trace-level investigation.
- →From LangSmith or similar hosted trace tools: re-point instrumentation at Phoenix via OpenTelemetry to keep the same span data without per-span billing.
- →From a homegrown eval script: import existing labeled examples as a Phoenix dataset and add LLM-as-judge evaluators alongside them.
- →From Phoenix Cloud to self-hosted OSS: run `uvx arize-phoenix serve` or Helm, then update the collector endpoint.
- ↗To Arize AX: keep the same OpenTelemetry instrumentation and point it at the managed endpoint to gain Signal, custom code evaluators, and enterprise controls.
- ↗To a managed rival: because tracing is standard OpenTelemetry, re-point the exporter without rewriting instrumentation.
- ↗To Arize AX Enterprise: step up for SSO, audit logs, HIPAA support, and self-hosted AX deployment.
Integrations
Resources & Guides
Tutorials & Learning
YouTube returned 6 videos for “Phoenix”, and we withheld 6: 6 could not be judged, because “Phoenix” is a single word that other videos use for other things. We are showing none, because we could not prove any of them are about Phoenix.
Tools that pair well with Phoenix
Common stack mates teams adopt alongside Phoenix, with the specific reason each pairing earns its keep.
Arize Phoenix
Arize Phoenix is open-source LLM observability: trace agent steps, run evals, and self-host your traces.
LangSmith
LangSmith is agent and LLM observability: trace, monitor, and evaluate AI agents in cloud, BYOC, or self-hosted.
Langfuse
Open-source LLM observability and agent evals: trace, evaluate, and improve production AI agents with MIT-licensed tooling.
Featured Head-to-Head Comparisons
Chroma vs Phoenix
If your priority is debugging and evaluating complex AI agent workflows, choose Phoenix for its deep trace visibility and LLM-as-judge evaluations. If you need a cost-effective, scalable vector search engine for RAG or semantic retrieval, Chroma’s serverless architecture and recent auto-ingest features make it the stronger pick. Both are open-source and freemium, but serve fundamentally different needs.
Agnost Ai vs Phoenix
If you need a fully managed, production-focused observability layer that catches the weird edge cases your evals miss, Agnost AI is the pick — but you'll pay undisclosed enterprise prices and get zero integration ecosystem. If you want free, open-source, self-hostable control with deep trace-level debugging, LLM-as-judge evaluation, and ghost-trajectory simulation, Phoenix wins hands-down. Go Phoenix unless you specifically require a commercial vendor's closed-box anomaly detection.
Phoenix vs Thefastest Ai
If you need to pick the fastest provider for a latency-sensitive chatbot, TheFastest.ai gives you free, daily-updated benchmarks across regions. If you're debugging or evaluating complex AI agent workflows — with full traces, LLM-as-judge scoring, and dataset creation — Phoenix is the open-source choice. They serve different problems: speed measurement vs. agent quality. Your pick depends on whether you're optimizing for latency or building reliable agents.
Neon vs Phoenix
Neon is a serverless Postgres platform for app builders who need auto-scaling, branching, and AI backend primitives. Phoenix is an open-source observability tool for AI agent debugging and evaluation. They are complementary: Neon provides the data layer, Phoenix provides the monitoring layer. Choose Neon if you need scalable Postgres with branching; choose Phoenix if you need to trace and evaluate AI agent behavior.
Alternatives to Phoenix
View all
Arize Phoenix
Arize Phoenix is open-source LLM observability: trace agent steps, run evals, and self-host your traces.
Frequently Asked Questions
Used Phoenix? Help shape our editorial sentiment research.