imp

LLM server for one RTX 5090. Qwen3.8-27B at 91.0-93.7 tok/s (vLLM 68.9-70.0, llama.cpp 61.8-63.0). Runs Qwen3.8-Flash-Next (123.6 GiB) on a 32 GB card at 87.89-98.57 tok/s. Built for agents: tool calls, long context, many streams at once.

GitHub

Official source

Start locally with the CLI guide