Alternatives to Petals

Decentralized, unfiltered AI inference on a peer-to-peer GPU network.

picoLLM is Picovoice's on-device LLM inference engine for running quantized language models locally on phones, embedded Linux, web, and microcontrollers.

Iris Android runs GGUF language models offline on your phone using llama.cpp — no server, no account, no data leaving the handset.

MAX Engine: OpenAI-compatible inference for open models, one stack across NVIDIA, AMD, Apple silicon, and Qualcomm.

Groq runs an inference cloud tuned for fast LLM inference on open-weight models, built on its LPU and LPX silicon.

DeepInfra delivers serverless AI inference on 100+ open models through one OpenAI-compatible API key.

Together Compute is an OpenAI-compatible inference API for serving open-weight AI models at published per-token rates.

SambaNova Cloud delivers fast, energy-efficient inference for open-weight models like MiniMax M2.7 and DeepSeek-V3.1 on its own RDU chips.

Free, open-source desktop app for running and fine-tuning RWKV RNN language models locally with effectively unbounded context.

GMI Cloud Inference Engine runs text, image, video, and audio models behind one OpenAI-compatible API on dedicated NVIDIA GPUs.

Parallax turns mismatched machines into one self-hosted LLM cluster for decentralized inference.

Wafer Pass swaps per-token inference billing for a flat-rate subscription on open models like GLM, Qwen, DeepSeek, and Kimi.

OnnxStream runs ONNX inference by streaming model weights from disk, so large models fit in tiny RAM budgets.

TensorRT-LLM is NVIDIA's open-source library for maximum-throughput LLM and visual generation inference on NVIDIA GPUs.

Run local HuggingFace models or route prompts to Claude, GPT and Gemini with your own API keys

BitNet (bitnet.cpp) runs 1-bit and ternary BitNet b1.58 LLMs on CPU with lossless 1.58-bit inference — no GPU needed.

Ollama runs open-weight LLMs locally or in the cloud with one command and an OpenAI-compatible API.

Cerebras inference runs GPT-5.6 Sol Ultrafast at up to 750 tokens/sec on wafer-scale hardware for latency-critical agents, code tools, and

Modular runs one AI inference stack — same model, container, and OpenAI-compatible API — across NVIDIA, AMD, TPU, Trainium, and Qualcomm

Etched builds custom inference silicon and rack-scale frontier inference clusters for trillion-parameter MoE prefill and decode.

Runware is a pay-as-you-go inference API for image, video, audio, 3D and LLM generation on one endpoint.

Chiplet-based AI inference accelerators — Rebel100 silicon, RebelCard, RebelServer, RebelRack and RebelPOD for enterprise LLM inference.

Mistral is the sovereign AI stack: frontier open-weight models you can self-host, fine-tune, or run in-region.

Run 15+ offline AI models on your phone for private, on-device chat, image, music and code generation.

SGLang is the open-source LLM serving framework built for high-throughput, low-latency inference on your own GPUs.

Run any GGUF AI model locally on Android with zero network permissions

Novita AI runs 200+ open-weight LLMs, image, audio and video models through one serverless API, plus GPU cloud and an Agent Sandbox.

Free, MIT-licensed local LLM inference for Apple Silicon Macs, with SSD-backed prefix caching and native OpenAI and Anthropic APIs.

Open-source LLM inference and serving engine with PagedAttention, continuous batching and an OpenAI-compatible API.

Pioneer is a self-improving inference API that routes every call to the right model and retrains itself from your production traffic.