Pmetal

Pmetal is a high-performance Apple Silicon framework for local LLM inference, fine-tuning (LoRA/QLoRA/DoRA), serving, quantization, model merging, and distributed training across Macs. Built as a Rust workspace with a zero-allocation MLX bridge, custom Metal and ANE paths, and TurboQuant long-context serving, it supports everything from pretraining to multi-Mac distributed training over Thunderbolt. The platform offers a 20-workspace terminal TUI, a 19-screen Tauri desktop GUI, CLI commands, and Rust/Python SDKs, all backed by the same job specifications and event stream. Pmetal targets power users and developers on macOS needing native, efficient ML workflows without cloud dependencies. What sets Pmetal apart is its deep integration with Apple's hardware—from M1 through M5—using tier-aware tuning, Metal 4/MPP dispatch, and a unified job/event substrate. It includes a KV cache compression technique (TurboQuant) that claims 4-6x compression, long-context discontinuous batch serving, and distributed training over Thunderbolt fabrics. The tool supports a wide range of model architectures (e.g., Qwen, Llama, DeepSeek, Mistral, Gemma, Phi, Cohere, Granite) and operations including SFT, DoRA, GRPO/DAPO, RLKD, distillation, and embedding training. While the homepage highlights powerful capabilities like continuous batching, structured outputs, and shared prefix caching, the documentation is sparse and some features may be experimental. Pmetal is designed for developers comfortable with Rust or Python and willing to experiment with emerging Apple Silicon ML infrastructure. Compared to alternatives like Ollama or LM Studio, Pmetal offers far deeper control over hardware and training, but has a steeper learning curve and narrower OS support. It's not a beginner's tool, but for those who need to squeeze every drop of performance from Apple Silicon, it's unmatched.