RWKV Runner

When you're evaluating local LLM runners, RWKV Runner doesn't get the attention it deserves. The 8MB desktop app is deceptively simple: it bundles inference, training, and fine-tuning behind a GUI, and it exposes an OpenAI-compatible API, which means you can wire it into anything that speaks that protocol. The big selling point is the architecture. RWKV's RNN design gives you linear-time inference and constant memory—no KV-cache—so context windows can stretch to what feels like infinite. For tasks like summarizing a whole codebase or analyzing a long legal document, that's a real advantage over Transformer-based models that choke on long inputs. In practice, you won't hit a context limit that matters. And the numbers are credible: 10,250+ tokens per second on an RTX 5090 for the 7B fp16 model is fast. Being able to fine-tune a 7B model on just 9GB VRAM is a standout capability—most tools need far more. If you're on a consumer GPU, that's a genuine differentiator. Where it bites: the ecosystem of pre-trained models is nothing like what you get with GPT or Llama. You're limited to RWKV variants, mostly RWKV-7 'Goose' and its predecessors. If you need a specific fine-tuned model for a niche task, you might not find it. Also, if you want a hosted service with an SLA, this isn't that—you're running everything locally, so you're on the hook for your own hardware and maintenance. Compared to something like Ollama, RWKV Runner is more specialized but includes training and fine-tuning, which Ollama doesn't offer natively. Compared to a cloud API, you get privacy and no per-token costs, but you lose the convenience and scale. We'd reach for RWKV Runner when you prioritize efficiency, infinite context, and full open-source freedom over ecosystem breadth. It's the right call