Mirai

The clearest thing Mirai gets right is scope discipline. Most on-device AI efforts start from a cloud model and shrink it; Mirai starts from the device constraint and builds up. That shows in the parts it ships. uzu is a Rust inference engine optimized for tensor multiplications and other operations on Apple silicon, operating at batch size of one—the only batch size that matters when a single user is waiting. lalamo handles optimization and conversion so a model can actually land on-device. The CLI lets you chat with models and serve them as a local API, which is the fastest path to a working prototype. The macOS app and models library fill in the day-to-day. Where Mirai earns credibility is research. The June 2026 quantization method targets the speed-quality frontier for local LLMs on Apple silicon directly, not as a paper exercise. The June 2026 sparse buffers for KV cache address memory pressure during inference, which is the real ceiling on device. July 2026's Trees from Marginals explores autoregressive drafting with factorized priors. Whether or not each one ships into your build, the direction is consistent and the open-source posture means you can verify claims rather than take them. Now the honest side. This is Apple Silicon only—iOS and macOS—with Android listed as "Soon." The docs state the constraint plainly, and the company has framed that constraint as a feature rather than a bug to fix. That is a defensible position, but it is still a hard boundary for any team with an Android or Windows roadmap. The skill bar is real: quantization and model conversion are not beginner territory, and the docs describe a setup where you are working with a runtime and conversion toolchain rather than pasting an API key. There is also no published price list; the pricing page routes to a contact form at contact@trymirai.com, so procurement needs a conversation. If your team wants a vendor-managed endpoint and SLAs tomorrow, that friction matters. The fit, then, is narrow and clear. You are building an AI feature where the 200ms loading state is the failure mode—parsing, validating, ranking, rendering in one loop on a device you control. You have ML capability in-house or you are willing to build it. You want the execution to be private by construction, because the inference never leaves the hardware. Mirai is a poor fit if you need one SDK that covers iOS, Android, and the web; if you want someone else to run the models; or if your workload is batch-shaped rather than interactive. That is not a criticism of the product so much as a description of who should stop reading here.