Features
Native inference server for Apple Silicon (M1-M4)
OpenAI and Anthropic compatible API endpoints
Speculative decoding (PLD, cross-attention drafter, MTP)
Adaptive gates to disable speculation on novel content
DeepSeek V4 Flash (284B) support on 96GB+ Macs
Agent mode and MCP tool calling
Photo editing with natural language
Image-to-video and talking-character video
Voice cloning and Krea-2 image generation
Style LoRAs and DiffusionGemma
Model browser with one-click download of GGUF/MLX
OpenAI-compatible API for model serving
Deploy 500+ open-source models
Write custom GPU kernels with Mojo
Zero dependency on PyTorch, CUDA, or ROCm
Single container under 700MB for self-hosted
Paged KV cache for memory efficiency
Quantization (bfloat16, float32)
Multi-node distributed inference
Model customization via PyTorch-like API
Hardware-agnostic (NVIDIA, AMD, Apple Silicon)
Mixture-of-Experts serving
max benchmark tool adapted from vLLM
MiniMax M3 open weights support
SOTA MoE serving on Modular Cloud
