lm-warden

Your own OpenAI-compatible LLM API on your own NVIDIA GPUs. vLLM and llama.cpp behind one control plane, one port and a browser UI: pull models from Hugging Face, issue API keys with per-key usage, see what the GPUs are doing. Self-hosted with Docker Compose; nothing reports home.

GitHub

Official source

Start locally with the CLI guide