PromptProof
Test your LLM prompts with statistical confidence
1 follower
Test your LLM prompts with statistical confidence
1 follower
PromptProof brings statistical rigor to prompt engineering. Confidence intervals tell you if gains are real, not noise. Test images, videos, and PDFs on major LLMs in one place. Auto-Refine optimizes prompts on your labeled data under a cost cap, returning the top 3 by accuracy and cost. Decision Lab compares adoption candidates on accuracy, stability and cost. In production, trace real inputs and label them to validate accuracy. Built by an ML engineer tired of "improvements" he couldn't prove.
This is the 2nd launch from PromptProof. View more
PromptProof Auto-Refine
Launching today
Stop hand-tuning prompts. Auto-Refine optimizes your prompt against your own labeled data, keeps spend under a hard cost cap, and returns the top 3 candidates ranked by accuracy and cost — so you ship measurable gains, not vibes.
Free Options
Launch Team
AWS Startup AdvisorPersonalized AWS guidance that meets you where you’re already building
Promoted
Hi Product Hunt! 👋
We launched PromptProof here in April to bring statistical rigor to prompt testing — confidence intervals instead of "it looks better."
Today we're launching Auto-Refine: instead of hand-tuning prompts and re-running experiments yourself, you point it at your labeled dataset, set a cost cap, and it searches for better prompts automatically. You get the top 3 candidates ranked by accuracy and cost, with the stats to prove the improvement is real.
It works across OpenAI, Anthropic, Google, and open models (via Together AI), and handles free-text outputs too, not just structured extraction.
Would love your feedback — especially on what trade-offs you'd want to control (accuracy vs. cost vs. latency).