APIMaster.ai
Detect fake LLM APIs & reveal the real model underneath
Start new thread
Chuang Ma

APIMaster.ai - Detect fake LLM APIs & reveal the real model underneath

by•
The original idea was simple: I kept wondering – Is this LLM API I’m paying for actually real?? So I built something to answer that question. How does the detector work? I extracted 300+ stylistic features from each model (Opus 4.8 loves saying “genuinely”, “honestly”, and using “I” a lot, for example). Then built a classifier that can tell you what model is actually running behind the API. Based on 1k+ real user tests, 41% of LLM APIs out there are fake.

Add a comment

Replies

Best
Chuang Ma
Maker
📌
Hey Product Hunt 👋 I built APIMaster.ai after getting tired of not knowing whether the "Claude Opus" I was paying for was actually Claude Opus. The obvious check — asking the model "what are you?" — is trivially fakeable. Any reseller can patch a system prompt to make GPT-3.5 swear it's Opus. So I built something that doesn't ask. It measures. The tool probes behavioral signatures: token-level decision patterns under ambiguity, response structure preferences, characteristic failure modes on adversarial inputs with noise injection. These survive system prompt overrides. To fake them convincingly, you'd have to actually run the real model — which defeats the whole point of cheating. What the data showed on day 1 (from r/SillyTavernAI, 19K views): ✅ Verified: api.navy (97%), nano-gpt.com (84%), openrouter.ai (84.7%), pi.tokenator.cloud (78%) 🔴 Fraud: api.airforce — serves GPT 5.4 at 100% confidence regardless of what model you request (Opus, Sonnet, Gemini — all GPT 5.4) 🟠 Suspicious: 9 out of 9 "cheap Opus" endpoints in my original batch — all showing probability mass leaking into haiku-4-5, deepseek-v4-flash, and other cheaper models The tool is free, MIT licensed, no API key storage (use a throwaway key anyway — that's good hygiene with any third-party tool, including this one). Happy to answer questions about the methodology — especially skeptical ones. The code is open if you want to audit it. Cheap != Cheat. But verify before you trust.