Features
MetalRT: custom Metal GPU kernels for Apple Silicon
QHexRT: full-stack NPU inference for Qualcomm Hexagon
Open-source SDKs: Swift, Kotlin, React Native, Flutter, Web
On-device inference by default with automatic cloud routing
Mirar: real-time vision agent with local pre-filtering
LLM, VLM, STT, TTS, and speech-to-speech inference support
Fleet operations and OTA model updates via console
Sub-10ms local inference latency
Cost-aware policy engine for routing
Benchmark-backed performance with methodology disclosure
Integration with OpenRouter (300+ models)
Integration with vLLM, Gemini, GPT-4o, Claude
Custom endpoint integration support
Distributed tracing for LLM agents
Capture prompts, retrievals, tool calls, outputs
Human annotations on traces
Create datasets from traces
Run experiments to compare changes
OpenTelemetry-native instrumentation
Self-host locally, Docker, or Kubernetes
Cloud instances with free tier
Vendor agnostic: any model or framework
3M+ monthly downloads, 9k+ GitHub stars
