Features
On-device LLM inference with X-Bit quantization (sub-4-bit)
No cloud dependency; all data stays on device
Real-time inference for voice and text applications
Integrates with Picovoice voice AI stack (wake word, STT, TTS)
Supports RAG (Retrieval-Augmented Generation) for document QA
Low latency and offline operation
Privacy-preserving: no data leaves the device
SDK for multiple platforms (Android, iOS, Linux, macOS, Windows, Web, Python)
Customizable model compression with picoCompression
Benchmarked against GPTQ, GGUF, and SpinQuant for accuracy/speed
Local model execution for offline inference
Connect online APIs: OpenAI (ChatGPT), Anthropic (Claude), Google (Gemini), Meta (Llama), Mistral AI, Alibaba (Qwen),
123+ HuggingFace models available
Memory feature (coming soon) for context persistence
Minimalist UI with dark mode
Open source under MIT license on GitHub
Cross-platform desktop app (Mac, Windows, Linux)
Version 0.8.3 with bug fixes and performance improvements
