Features
Hierarchical traces with sessions and agent graphs
Prompt versioning and deployment via labels
Playground for testing prompts on production inputs
LLM-as-a-judge and heuristic evaluators
Human annotation queues with keyboard shortcuts
Experiments with datasets and side-by-side comparisons
Cost, latency, and quality dashboards with monitors
Monitors that alert via Slack, webhooks, GitHub Actions
Ask AI filter search bar (AWS Bedrock, zero data retention)
Multi-modal datasets (images, audio, video, documents)
Web callouts to trigger external workflows
MCP server for AI agent integration
Scores API v3 with cursor-based pagination
SCIM API and SSO enforcement (Enterprise)
Self-hosting via Docker, Kubernetes, Terraform
Distributed tracing for LLM agents
Capture prompts, retrievals, tool calls, outputs
Human annotations on traces
Create datasets from traces
Run experiments to compare changes
OpenTelemetry-native instrumentation
Self-host locally, Docker, or Kubernetes
Cloud instances with free tier
Vendor agnostic: any model or framework
3M+ monthly downloads, 9k+ GitHub stars
