convy

Conversation tests for AI agents: a simulated user talks to your agent through its API, a judge checks every claim, and convy reports pass rate, tokens, time and what changed between builds.

GitHub

Official source

Start locally with the CLI guide