FlakyBot

Hey Product Hunt! We’re Spriha and Ankit, co-founders of MergeQueue - we’re back with our second product: FlakyBot! Unreliable (or “flaky") tests are tests that raise false alarms or produce different results when run multiple times against the same build. As you’d expect, flaky tests erode trust in a test’s results, which in turn erode trust in the code being submitted. FlakyBot helps teams manage test-flakiness. It automatically monitors, identifies, and suppresses flaky tests so that developers are better able to trust their test results. It also allows for automatic retries of tests that are known to be flaky. How it works: FlakyBot is a plugin that integrates into your CI system. Once integrated, it analyzes test artifacts from your CI and compares them to historical test-data to identify flaky failures from true failures. The bot then produces a final “flake-normalized” report for each test run. This can be used as a status check on GitHub, Bitbucket or GitLab, to ensure developers only see true failures in their testing workflow. If you’ve experienced problems with unreliable tests, sign up for access at FlakyBot.com and send me an email at spriha@mergequeue.com with the subject line “Product Hunt promo” for early access and a free month!

This is awesome, congrats on the launch! ?makers is a flaky test defined as a test that fails often, or is it more complex than that?

Really cool work guys, congrats!

Super helpful product, congrats on launching!