opod-core

Self-hosted AI gateway for the GPUs you already own: one OpenAI-compatible endpoint across many machines. Engine-agnostic (Ollama, vLLM, SGLang, llama.cpp, MLX-LM), load-aware routing, one model sharded across nodes, API keys, Prometheus + OpenTelemetry. Go, Apache-2.0, no telemetry.

GitHub

Official source

Start locally with the CLI guide