Features
Native inference server for Apple Silicon (M1-M4)
OpenAI and Anthropic compatible API endpoints
Speculative decoding (PLD, cross-attention drafter, MTP)
Adaptive gates to disable speculation on novel content
DeepSeek V4 Flash (284B) support on 96GB+ Macs
Agent mode and MCP tool calling
Photo editing with natural language
Image-to-video and talking-character video
Voice cloning and Krea-2 image generation
Style LoRAs and DiffusionGemma
Model browser with one-click download of GGUF/MLX
Distributed model training across GPU clusters
Multimodal data curation (video, images, text, audio)
Batch embedding generation with Sentence Transformers
Post-training LLM inference with vLLM and SGLang
Elastic scaling with last-mile data preprocessing
Fine-grained hardware allocation (CPU, GPU, TPU, NVL72)
Multi-cloud orchestration across GPU providers
Ray-native distributed object store and RDMA transport
Automatic cluster provisioning and scaling
GPU observability and advanced monitoring
Agent-first experience with Python APIs
Serverless execution with Python decorators
Bring Your Own Cloud (BYOC) deployment
Integration with PyTorch, vLLM, SGLang, XGBoost
On-premises deployment support via BYOC
