I built an AI business auditor, then made it strict enough to fail me.🐝

I built an AI business auditor, then made it strict enough to fail me. But still shipped it anyways.🐝

I built 🐝MoltyBeeAI™ — seven agents analyse a business from a structured intake and return a scored diagnosis: an AI Leverage Score across four dimensions, quantified revenue leaks, a bottleneck map, an agent deployment blueprint, and a 30/60/90 roadmap. Minutes, not weeks. $98.

The interesting part isn't the agents. It's what happened when I tried to make the score mean something.

v1 scored in prose. An agent read your intake and wrote a paragraph explaining why you got 69/100. Authoritative-sounding and completely unfalsifiable. Fine for a one-off report, fatal if you intend to build benchmarks on it.

So I replaced it with a formal rubric: 24 criteria across 4 dimensions, each with an anchor — a specific checkable fact that earns the points — plus partial-credit rungs, evidence types, and a verification status. Versioned, so the rubric can evolve without corrupting historical scores.

Then I ran it on my own company. 69 → 34.

Almost all of the 35-point gap was trajectory credit: points for a pricing tier "in development," for an LTV model with no cohort data behind it, for positioning that existed in my head and nowhere else. Plans score zero on a rubric measuring what operates today.

I kept the 34 and published it.

Then the verification problem. Every criterion is tagged verified or self_reported, and only verified criteria count toward benchmarks. Sounds rigorous — except nothing could ever become verified, because the intake collected claims, not evidence. Every business hit the same benchmark ceiling of 16/100. Percentiles on that distribution would have been noise in a suit.

Evidence intake took three attempts:

Does the URL resolve? Built it, tested it, realised it verifies nothing. Paste your homepage into every field and pass.

Fetch the page, let the model judge. Better — but an LLM asked "does this support the claim?" is a system you can talk past.

What shipped: a deterministic marker floor before any model reasoning. A checkout page must contain a currency pattern AND a purchase action. A case study must contain measured-outcome language AND named attribution. Only artifacts clearing the mechanical floor become eligible for verification — the model can still refuse, but it cannot grant beyond the eligible list.

real checkout page: PASS

generic homepage: BLOCKED — missing: a stated price, a purchase mechanism

coming-soon page: BLOCKED — missing: a stated price, a purchase mechanism

real case study: PASS

vague testimonials: BLOCKED — missing: measured outcome language, attribution

Then it caught me.

First live evidence run, my benchmark score jumped 16 → 37. Ceiling broken. I was pleased for about ninety seconds, then read the flags: I'd pasted moltybeeai.com into four separate evidence fields. All four passed — my pricing, FAQ, positioning and metrics genuinely do live on one long homepage. But the system had flagged same_url_multiple_fields, and I'd built flags as advisory rather than binding.

The uncomfortable part: the system can't distinguish my honest one-page site from someone lazily pasting the same link four times. Identical evidence, opposite intent. When a rule can't tell honest from dishonest, the correct move is to refuse both and give the honest party a path to prove it.

I shipped duplicate-URL exclusion. My own score dropped again.

Design decisions that might interest people here:

The model never does arithmetic. It reports per-criterion points; the server computes every sum, cap and band. LLM addition is unreliable and the score is the product.

Verdicts, not vibes.™

Two scores per audit. Displayed, and benchmark_eligible computed from verified criteria only. A database CHECK constraint makes it impossible for the benchmark number to exceed the displayed one.

Self-reported points are capped at 4 per dimension in the benchmark layer. You can claim anything; it won't move you in the percentiles.

Rubric versions are pinned to every record and percentiles compute only within version cohorts, so a change in the ruler is never mistaken for a change in the business.

Trajectory thresholds are calibrated against measured model noise. Four runs on an unchanged business gave 34, 34, 36, 34 — so ±2 is noise, and the improving/declining threshold is 3.

Published ceiling: only 9 of 24 criteria are verifiable by a public artifact. Internal metrics — LTV cohorts, funnel rates, time audits — stay self-reported for now as we work on creating & adding more value for you guys. Max benchmark-eligible score is 57/100, disclosed on the intake form, & you can check it out now at www.moltybeeai.com . A ceiling everyone shares is fair; a hidden one isn't.

Verdicts, not vibes.™

That's our whole product.

And the craziest part of this entire build. Built and shipped my startup entirely from my smartphone, which I'd never recommend but it turned out to be possible for me.

I certainly have God to thank.

Happy to go deep on why I think prose output is the biggest unforced error in most AI products right now — the AI agents were producing genuinely useful intelligence that nothing could compute on until I forced it into records.

And that's what I'm excited about the most, but also, what have you discovered so far running 🐝MoltyBeeAI™ Swarm Audits™ v3.2 on your business?

Go try the product out, now. At www.moltybeeai.com 🐝

🐝Thank you,

— Founder & Operator, 🐝MoltyBeeAI™

Sihle Dimaza

•Building 🐝MoltyBeeAI™ in public.