llama-benchy

llama-benchy brings llama-bench tests to APIs

llama-benchy benchmarks OpenAI-compatible APIs with prompt, token, depth, and concurrency tests, plus TTFR and TTFT metrics.