Features
MetalRT: custom Metal GPU kernels for Apple Silicon
QHexRT: full-stack NPU inference for Qualcomm Hexagon
Open-source SDKs: Swift, Kotlin, React Native, Flutter, Web
On-device inference by default with automatic cloud routing
Mirar: real-time vision agent with local pre-filtering
LLM, VLM, STT, TTS, and speech-to-speech inference support
Fleet operations and OTA model updates via console
Sub-10ms local inference latency
Cost-aware policy engine for routing
Benchmark-backed performance with methodology disclosure
Integration with OpenRouter (300+ models)
Integration with vLLM, Gemini, GPT-4o, Claude
Custom endpoint integration support
Hierarchical LLM traces with cost/latency filtering
LLM-as-a-judge evaluation and heuristic functions
One-click prompt deployment and rollback
Playground for side-by-side model/input testing
Experiments with test case comparison
Human annotation and golden dataset creation
Cost and latency dashboards with alerts
Monitors and alerts (Slack, webhooks, GitHub Actions)
Full-text search (Cloud rollout)
Code evaluators (Python/TypeScript)
Langfuse Assistant (natural-language queries)
Multi-modal datasets (images, audio, video, documents)
OpenTelemetry-native instrumentation
Python and TypeScript native SDKs
REST APIs and S3 blob storage export
