Features
Hierarchical traces with sessions and agent graphs
Prompt versioning and deployment via labels
Playground for testing prompts on production inputs
LLM-as-a-judge and heuristic evaluators
Human annotation queues with keyboard shortcuts
Experiments with datasets and side-by-side comparisons
Cost, latency, and quality dashboards with monitors
Monitors that alert via Slack, webhooks, GitHub Actions
Ask AI filter search bar (AWS Bedrock, zero data retention)
Multi-modal datasets (images, audio, video, documents)
Web callouts to trigger external workflows
MCP server for AI agent integration
Scores API v3 with cursor-based pagination
SCIM API and SSO enforcement (Enterprise)
Self-hosting via Docker, Kubernetes, Terraform
Trace visibility for agent steps (prompts, retrievals, tool calls, outputs)
LLM-as-judge evaluation for relevance, toxicity, quality scoring
Dataset creation from traces for reproducible testing
Experiment management and regression benchmarking
Built-in Prompt IDE for iterative prompt optimization
Self-hosted deployment on local, Docker, Kubernetes
Phoenix Cloud managed hosting option
Vendor-agnostic support for any model/framework
Native OpenTelemetry integration
OpenInference specification for LLM telemetry
Human annotation and automated labeling
Ghost trajectories to simulate alternative agent paths
Eval-as-you-test for early quality feedback
One-click integration with LlamaIndex
