Axon

Fast, cheap LLM inference engine written from scratch in Rust + CUDA. Runs GGUF models (Qwen, Llama, Gemma, Mistral, MoE) on CPU and NVIDIA GPUs, with an OpenAI / Responses / ollama-compatible server. No ggml, no BLAS.

GitHub

Official source

Start locally with the CLI guide