cli-modelarium

Statistically rigorous LLM comparison CLI for terminal-first developers. Compare 12 cloud providers + local models side-by-side with bootstrap confidence intervals, significance testing (paired t-test, McNemar), LLM as judge, hallucination detection, and CI/CD assertions.

GitHub

Official source

Start locally with the CLI guide