async-batch-llm

Provider-agnostic framework for high-throughput LLM processing with async workers, automatic retries, rate limiting, and intelligent validation recovery.

GitHub

Official source

Start locally with the CLI guide