Features
Automatic LLM tracing via OpenTelemetry
Built-in evaluations: faithfulness, relevance, safety
Custom evaluator training with annotation
Real-time monitoring dashboard
Evaluation dashboard with test runs
CI/CD integration for automated evaluation
Prompt management and registry
LLM hub proxy for smart routing
Granular latency and token usage tracking
Cost attribution per user or feature
On-prem and air-gapped deployment
SOC 2 and HIPAA compliance
One-line code setup for tracing
Supports Python, TypeScript, Go, Ruby
Support for 20+ LLM providers
Trace visibility for agent steps (prompts, retrievals, tool calls, outputs)
LLM-as-judge evaluation for relevance, toxicity, quality scoring
Dataset creation from traces for reproducible testing
Experiment management and regression benchmarking
Built-in Prompt IDE for iterative prompt optimization
Self-hosted deployment on local, Docker, Kubernetes
Phoenix Cloud managed hosting option
Vendor-agnostic support for any model/framework
Native OpenTelemetry integration
OpenInference specification for LLM telemetry
Human annotation and automated labeling
Ghost trajectories to simulate alternative agent paths
Eval-as-you-test for early quality feedback
One-click integration with LlamaIndex
