Alternatives to Mini Infer

vLLM is the open-source high-throughput LLM inference and serving engine with PagedAttention and an OpenAI-compatible API.

MAX Engine serves open-source LLMs through an OpenAI-compatible API on NVIDIA, AMD, and Apple silicon with no CUDA or PyTorch dependency.

SGLang is the open-source LLM inference engine for high-throughput, low-latency serving across NVIDIA, AMD, TPU, and NPU hardware.

NVIDIA's open-source library for optimizing LLM and visual generation inference on NVIDIA GPUs with specialized kernels and a Python API.

DeepInfra is a serverless inference cloud serving 100+ open models — DeepSeek-V4-Flash-0731 at $0.06 per 1M input tokens — through one OpenAI-compatible API

Microsoft's open-source 1-bit LLM inference framework for fast, lossless CPU and GPU deployment

Together Compute is an AI-native cloud for running open-source models — serverless inference, batch jobs, model shaping, and GPU clusters in one platform.

Predibase is a managed platform for fine-tuning and serving open-source LLMs, now part of Rubrik.

Groq is an inference neocloud built for sub-200ms LPU inference — fast open-weight model serving for real-time chat, voice, and agent workloads.

Apache-2.0 framework for fine-tuning 100+ LLMs and VLMs through a zero-code CLI and the LlamaBoard web UI

Custom RDU hardware for fast inference on open models, sold as racks and cloud capacity.

Multimodal AI inference platform with OpenAI-compatible APIs, dedicated GPUs, and day-zero frontier models like Qwen3.8-Max and Kimi K3.

Open-source Kubernetes operator that deploys and autoscales LLMs, embeddings, reranking, and speech-to-text with an OpenAI-compatible API.

Open-source distributed model serving framework that turns mismatched machines into one self-hosted AI cluster

Flat-rate inference on open LLMs with continual optimization for agentic coding and production workloads.

AI-native GPU cloud for large-scale model training, reinforcement learning, and low-latency inference on NVIDIA's newest hardware.

Fine-tune and run LLMs locally with Unsloth — custom CUDA kernels cut VRAM and speed up training on your own GPU.

Anyscale Endpoints runs distributed training, batch inference, and multimodal data curation on managed Ray GPU clusters.

Cerebras delivers ultra-fast AI inference on wafer-scale hardware for latency-critical agents and apps.

Unified AI inference stack from GPU kernel to API endpoint, portable across NVIDIA, AMD, TPU, Trainium, and Qualcomm silicon.

Verda is a European full-stack AI cloud for on-demand NVIDIA GPU instances, instant InfiniBand clusters, and serverless inference.

Frontier inference clusters for extreme-scale transformer workloads.

Runware: one AI inference API for image, video, audio, 3D and LLMs at PAYG rates

Rebellions builds chiplet-based AI inference accelerators — the Rebel100 chip plus RebelCard, RebelServer, RebelRack and RebelPOD systems — for enterprises and

Mistral sells frontier open-weight AI models, long-horizon Vibe agents, and document intelligence you can self-host or run on EU

Decentralized, unfiltered AI inference on a peer-to-peer GPU network.

Pioneer is a self-improving inference API that routes every call to the right model and retrains itself from your production traffic.

Mesh LLM splits giant open-weight models across your own GPUs and serves them behind one local OpenAI-compatible API.

Unified LLM API gateway: one OpenAI-compatible endpoint for 96+ text, image, and video models at up to 90% off list price.

Reconfigurable XPU chips that run AI training and inference on a fraction of the energy.