Features
One-click deploy to Cloudflare Workers
Built-in authentication and rate limiting
Caching of AI responses for faster repeat calls
Supports multiple Gemini model versions
Integration with Apple Shortcuts via URL action
Automatic retry on transient errors
Logging and analytics for API usage
Customizable system prompts
Streaming responses for real-time output
CORS-enabled for cross-origin requests
OpenAI-compatible API for model serving
Deploy 500+ open-source models
Write custom GPU kernels with Mojo
Zero dependency on PyTorch, CUDA, or ROCm
Single container under 700MB for self-hosted
Paged KV cache for memory efficiency
Quantization (bfloat16, float32)
Multi-node distributed inference
Model customization via PyTorch-like API
Hardware-agnostic (NVIDIA, AMD, Apple Silicon)
Mixture-of-Experts serving
max benchmark tool adapted from vLLM
MiniMax M3 open weights support
SOTA MoE serving on Modular Cloud
