WhyLab

Find why your ML model fails, prove it, repair it

3 followers

Find why your ML model fails, prove it, repair it

3 followers

WhyLab is an AI incident investigator built on GPT-6 Astra. Upload your model's predictions and Astra hunts for the root cause — ranking hypotheses, running falsification experiments on held-out data, and proposing a fix it then re-tests. No guesses presented as facts.

Free

Launch tags:User Experience•Education•Artificial Intelligence

Launch Team / Built With

AWS Startup AdvisorPersonalized AWS guidance that meets you where you’re already building

Promoted

OpenAI

GPT-6 Astra Challenge

What became possible in your product with Astra that was not practical before?

Maker

Before Astra, WhyLab's diagnostics were separate tools a user had to run and interpret themselves — the investigative reasoning (what to check next, which hypothesis to trust) still lived in the user's head, and didn't scale or stay consistent. Astra's async tool calling, function calling, and Structured Outputs let WhyLab hand over the investigation itself, not just the explanation. Astra plans which diagnostics to run, proposes falsifiable root-cause hypotheses from the evidence, and orchestrates the tests needed to confirm or reject them — then verifies a proposed repair the same way. That multi-step, tool-orchestrating loop — plan, gather evidence, hypothesize, test, repair, verify — wasn't practical with a single-shot LLM call. It needed a model that can genuinely steer a multistep technical workflow, which is what made a real ML incident investigator possible instead of a dashboard with an AI summary bolted on.

Report

2mo ago

Maker

📌

Hey Product Hunt 👋 I built WhyLab because I kept seeing the same failure mode in ML projects: a model looks great on a held-out test set, ships, and then fails in ways nobody can explain — and the "explanation" that eventually gets written up is usually just a plausible story, not a proven one. For the GPT-6 Astra Challenge, I rebuilt WhyLab around a rule I wouldn't let myself break: Astra is allowed to propose hypotheses, but it is never allowed to be the thing that confirms them. Every claim WhyLab shows you is backed by a statistical test run on data Astra never saw when it made the guess. If a hypothesis doesn't survive that test, WhyLab says so — "insufficient evidence" is a real, displayed outcome, not a bug. A few things I'm proud of under the hood: Astra orchestrates real diagnostic tools via function calling — it's planning an investigation, not narrating a dashboard Every hypothesis test uses proper stats (confidence intervals, significance testing, correction for testing multiple hypotheses) with hard sample-size floors The Repair Lab always shows before/after on the same held-out data, including metrics that got worse — no cherry-picking the win Would love feedback from anyone who's fought this exact problem in production. And if you've got a favorite "my model was great until it wasn't" war story, I want to hear it in the comments.

Report

2mo ago

Pros

Cons

Reviews

No reviews yetBe the first to leave a review for WhyLab