BodhiApp

Bodhi App is a self-hosted AI gateway that puts local GGUF inference and cloud API providers behind one authenticated server you run yourself. Local models execute through llama.cpp with GPU acceleration, and the vendor cites an 8-12x speedup with GPU acceleration over CPU-only inference. You pull weight files straight from HuggingFace with background downloads that track progress and auto-resume, then serve them through a built-in chat UI with markdown, model settings, thinking-model chain-of-thought, and real-time token streaming over Server-Sent Events. The gateway also proxies OpenAI, Anthropic, and Gemini traffic using your own credentials. Anthropic OAuth support lets you authenticate with a Bearer token drawn from a Claude.ai or Anthropic Console subscription instead of a metered API key. Existing client libraries connect without code changes because the server speaks OpenAI Chat Completions, Responses, and Embeddings plus Anthropic Messages and Gemini wire formats. The Ollama compatibility layer is now marked deprecated, with the vendor steering users toward the OpenAI, Anthropic, and Gemini surfaces. MCP integration lets models discover and invoke external tools mid-conversation. You get per-user MCP instances, three auth methods (Header, OAuth2 preregistered, OAuth2 Dynamic Client Registration), a playground for testing servers and individual tools, and per-conversation tool whitelists. Model aliases bind a GGUF file to 12+ inference parameters so you switch configurations without restarting. There is also a JS SDK with React hooks, an OpenAPI/Swagger UI, and scope-based API tokens hashed with SHA-256. For teams, Docker deployment ships OAuth2 PKCE + JWT auth, four roles (User, PowerUser, Manager, Admin), a self-service access request workflow with admin approval and audit trail, and app access consent controls. It suits developers who want inference and data on their own hardware and teams needing real multi-user auth rather than a single-user local