Features
LLM agent observability with OpenTelemetry tracing
Automatic issue detection in traces (March 2026)
Multimodal tracing for images, audio, and files (April 2026)
50+ built-in evaluation metrics and LLM judges
Prompt Registry with versioning and optimization
AI Gateway for unified LLM API access with guardrails (April 2026)
Agent Server for one-command production deployment
Role-Based Access Control (RBAC) with Admin UI (May 2026)
Automatic trace archival to object storage (May 2026)
One-click coding agent onboarding (May 2026)
Hermes Agent support (May 2026)
Experiment tracking and hyperparameter tuning
Model Registry with lineage and deployment
Model evaluation and comparison
Integration with 100+ tools and frameworks
Trace visibility for agent steps (prompts, retrievals, tool calls, outputs)
LLM-as-judge evaluation for relevance, toxicity, quality scoring
Dataset creation from traces for reproducible testing
Experiment management and regression benchmarking
Built-in Prompt IDE for iterative prompt optimization
Self-hosted deployment on local, Docker, Kubernetes
Phoenix Cloud managed hosting option
Vendor-agnostic support for any model/framework
Native OpenTelemetry integration
OpenInference specification for LLM telemetry
Human annotation and automated labeling
Ghost trajectories to simulate alternative agent paths
Eval-as-you-test for early quality feedback
One-click integration with LlamaIndex