Gpustack

GPUStack is an enterprise AI infrastructure platform that unifies Model-as-a-Service (MaaS) and GPU-as-a-Service (GPUaaS) under a single control plane. It enables organizations to deploy, govern, and scale LLMs and GPU compute on any hardware — on-premise, cloud, or hybrid, transforming any GPU into a high-performance token factory. The platform automates the entire inference workflow: connect model sources (Hugging Face, ModelScope, local files), auto-select the best inference engine (vLLM, SGLang, llama.cpp, TensorRT-LLM, MindIE) for your hardware, and scale across nodes with distributed inference (tensor/pipeline parallel, Ray). It serves models via OpenAI-compatible and Anthropic-compatible endpoints, making it a drop-in replacement for existing AI applications. GPUStack supports a wide range of GPUs including NVIDIA, AMD, Ascend, T-Head, Hygon, MetaX, Moore Threads, Cambricon, and Iluvatar. It provides built-in observability (Prometheus/Grafana), RBAC, SSO, token quotas, usage analytics, and metering/billing. The platform boasts performance gains like +135% throughput on GLM-4 with 8x H200 and -63% latency on Qwen3-8B. Unlike fully managed cloud services, GPUStack is self-hosted and open-source (v2.2 as of July 2026), with an Enterprise edition for additional governance and support. It's ideal for AI platform teams needing to manage heterogeneous GPU fleets on-premises or hybrid, offering flexibility and control without vendor lock-in.