Cactus

Cactus is best understood as an inference runtime plus a small-model research shop, not an app builder. Three layers do the work: the Cactus Engine (OpenAI-compatible APIs for C/C++, Swift, Kotlin, Flutter, with tool calling, auto RAG, NPU acceleration, INT4 quantization, and hybrid handoff), Cactus Graph (a zero-copy, PyTorch-like graph for custom models optimized for RAM and lossless weight quantization), and Cactus Kernels (ARM SIMD kernels tuned per-vendor with custom attention kernels, KV-cache quantization, and chunked prefill). That is a real engineering stack, not a prompt wrapper. Strengths. The published INT8 benchmarks are unusually specific by industry standards — Mac M4 Pro at 582/77 tps on LFM2.5-1.2B with 76MB RAM, iPhone 17 Pro at 300/33 tps with 108MB, Galaxy S25 Ultra at 226/36 tps but 1.2GB RAM. That last number is the honest part: the same model that fits in 76MB on Apple silicon takes 1.2GB on Snapdragon, which tells you NPU acceleration is not uniform and you need to benchmark your actual target devices. The .cact format is documented to the byte level (196-byte header, nameless tensor directory, 2.125 bits per weight, a 20-line parser in the blog), which matters if you are auditing memory-mapped loads at boot. Needle's architecture is also unusual and worth reading about directly: 70.8M of its 121M parameters live in hashed n-gram tables rather than feed-forward layers, so a token reads 30 rows and multiplies none. Weaknesses. This is SDK-first and hardware-dependent. You will write integration code, and your performance ceiling is set by the processor in the target device, not by Cactus. Cactus-Quantised models at roughly 2-bit are the default shipping path, and while the quantized artifacts are what makes 8–29 MB possible, they are not the same as running a full-precision model. Needle-family models are deliberately small — they are for routing, transcription, and tool calls, not for open-ended reasoning. If your product needs frontier-class answers on every single request, hybrid routing does not solve that; it just makes the inexpensive requests inexpensive. Where it fits. Screenless and always-on hardware, where the interaction is broken by a network round trip — the Pebble Index Ring is the public reference, and Raspberry Pi's own CEO has publicly run Needle on a Pi 5 mapping plain English to local Python actions offline. Also fits teams trying to move routine inference off cloud APIs to control spend and latency. Where it doesn't. Products that need prebuilt UI components, teams without engineering capacity to integrate an SDK and tune routing thresholds, and anything requiring very large context windows handled entirely on-device.