Hi, all! 👋 I built PromptProof out of a real pain I hit while developing a PDF extraction feature powered by LLMs. The prompt worked perfectly on the sample documents I was given. But once real user data came in, edge cases emerged. When I fixed those, the prompts didn't work well on the sample documents. It became a game of whack-a-mole with no way to measure if I was actually moving forward. Then came the real challenges: supporting new document formats without regressions, improving latency, upgrading LLM models — and each time, I had no reliable way to verify that the extracted data was still correct, not just that the API call succeeded. I realized this isn't unique to my project. Every team building LLM-powered features lives with this uncertainty. That's why I built PromptProof — to bring the kind of statistical confidence to prompt engineering that software engineers already expect from their test suites. Would love to hear from anyone who's faced the same pain. What's your biggest challenge when iterating on prompts in production? 🙏

PromptProof

Test your LLM prompts with statistical confidence

Test your LLM prompts with statistical confidence