Alternatives to Mlc Llm

Free, open-source desktop app for running and fine-tuning RWKV RNN language models locally with effectively unbounded context.

Predibase: managed fine-tuning and multi-LoRA serving for open-source LLMs, now inside Rubrik.

Ollama runs open-weight LLMs locally or in the cloud with one command and an OpenAI-compatible API.

LM Studio runs open-source LLMs locally with Bionic, an agent for coding, documents and automation.

GMI Cloud Inference Engine runs text, image, video, and audio models behind one OpenAI-compatible API on dedicated NVIDIA GPUs.

Free open-source local AI server that runs LLMs, image, music, video, and 3D generation on your own Apple Silicon Mac.

Open-source, local-first desktop AI client that connects to multiple model providers and keeps your data on your machine.

Novita AI runs 200+ open-weight LLMs, image, audio and video models through one serverless API, plus GPU cloud and an Agent Sandbox.

Atomic Chat runs 1,000+ local LLMs offline on your own hardware — free, open-source, no account, no rate limits.

Open-source LLM inference and serving engine with PagedAttention, continuous batching and an OpenAI-compatible API.

PrivateGPT is the open-source on-premise RAG framework for local document Q&A — 57k GitHub stars, zero cloud calls.

A free, open-source desktop chat client that runs your own OpenAI, Azure OpenAI, Google, and Replicate API keys from one window.

Open-source JavaScript library that runs small LLMs like Llama 3.2 1B Instruct inside the browser via WebAssembly and WebGPU.

MAX Engine: OpenAI-compatible inference for open models, one stack across NVIDIA, AMD, Apple silicon, and Qualcomm.

TensorRT-LLM is NVIDIA's open-source library for maximum-throughput LLM and visual generation inference on NVIDIA GPUs.

Groq runs an inference cloud tuned for fast LLM inference on open-weight models, built on its LPU and LPX silicon.

SambaNova Cloud delivers fast, energy-efficient inference for open-weight models like MiniMax M2.7 and DeepSeek-V3.1 on its own RDU chips.

SGLang is the open-source LLM serving framework built for high-throughput, low-latency inference on your own GPUs.

Enclave runs open-source AI models locally on your iPhone and Mac for private, fully offline AI chats.

Olares is an open-source personal AI cloud OS for running local LLMs, agents, and self-hosted apps on your own hardware.

Open-source Kubernetes operator for self-hosted LLM, embedding, reranking, and speech-to-text inference with an OpenAI-compatible API.

picoLLM is Picovoice's on-device LLM inference engine for running quantized language models locally on phones, embedded Linux, web, and microcontrollers.

DeepInfra delivers serverless AI inference on 100+ open models through one OpenAI-compatible API key.

BitNet (bitnet.cpp) runs 1-bit and ternary BitNet b1.58 LLMs on CPU with lossless 1.58-bit inference — no GPU needed.

Modular runs one AI inference stack — same model, container, and OpenAI-compatible API — across NVIDIA, AMD, TPU, Trainium, and Qualcomm

Together Compute is an OpenAI-compatible inference API for serving open-weight AI models at published per-token rates.

Mistral is the sovereign AI stack: frontier open-weight models you can self-host, fine-tune, or run in-region.

Free, MIT-licensed local LLM inference for Apple Silicon Macs, with SSD-backed prefix caching and native OpenAI and Anthropic APIs.

LFM2.5 is Liquid AI's open-weight foundation model family for on-device AI — native text, vision, and audio inference on CPU, GPU, or NPU.

Mesh LLM splits giant open-weight models across your own GPUs and serves them as one local OpenAI-compatible endpoint on localhost:9337.