Hey Product Hunt 👋 I built BYOK Multi-LLM Benchmarker because I kept bouncing between OpenAI, Anthropic, Gemini, and Groq's playgrounds just to see which model actually handled a prompt better or faster — and none of the existing tools let me compare them side by side without routing my keys through some third-party server. So this is a Chrome side panel: you paste in your own API keys (BYOK), pick the models you want to test, run one prompt, and get the responses plus time-to-first-token, total latency, and token counts laid out next to each other. Everything goes straight from your browser to the vendor's API — no proxy, no logging, no backend at all. It started as a quick script I used for my own work, then I turned it into an extension once I realized how often I needed this. Would love feedback, especially on which providers/models to add next (Mistral and local Ollama models are on my radar). Happy to answer any questions!

BYOK Multi

Compare GPT, Claude, Gemini & Groq — your keys, your data

Compare GPT, Claude, Gemini & Groq — your keys, your data