AI Model Benchmarking for Speed, Cost and Quality

★ 7.4 · vibe-coding

benchmark-models is a Claude Code skill that runs the same prompt or skill through Claude, GPT (Codex CLI), and Gemini side by side, then compares latency, tokens, cost, and tool-call counts across providers. All deterministic work lives in `scripts/model_benchmark.py` — pure Python stdlib, no SDKs — while the skill acts as an interactive wrapper: pick a prompt, preview provider auth with a dry-run, confirm which models to include, run, and interpret results. An optional Anthropic-API judge can score output quality for roughly $0.05 per run; it never auto-enables, since it costs real money. The final summary identifies the fastest, cheapest, and highest-quality option and names the tradeoffs explicitly. Use it when choosing a model for a specific task based on data rather than intuition, for cross-model comparisons, or a "model shootout".