Intercom — Startups get 90% off Intercom + 1 year of Fin AI Agent free
Startups get 90% off Intercom + 1 year of Fin AI Agent free
Promoted
Maker
📌
Hello Product Hunt! 👋
I've been working on a product that helps teams manage flaky tests. If you or your team ever feel like you need to spend more time re-running tests on pull requests, this product was made for you!
The app's main feature is the 'mute' feature. It allows you to silence a flaky test without any code changes. A muted test continues to run on your CI, collecting errors, but it won't fail your entire build until it's unmuted (which happens automatically after a week by default). This way, your team can maintain control without being overwhelmed by forgotten skipped tests.
Besides the mute feature, the product has all the other bells and whistles you'd expect: statistics, graphs, breakdown per test, error logs, etc.
I've also tried to make the setup as simple as possible, so all you need to do is install a package and add an environment variable in your CI.
I focused on Python 🐍 for the initial launch, but I'll add support for other languages and frameworks based on feedback.
This is my first product launch on Product Hunt, so I appreciate any feedback, thank you 🙏
Does it work as apart of a GitHub actions pipeline?
Report
Maker
@sentry_co Hey, yes, it does It should run in any CI just as long as you have the flakytest plugin pip package installed and the FLAKYTEST_SECRET_TOKEN env variable set.
I haven't gotten around to writing instructions for other CI systems yet, but if you let me know what you are using I'll make sure to prioritize writing docs for that!
@anze3db I think the most common use case would be Github actions. I use that for my remote test pipeline. Amongst others
Report
Maker
@sentry_co I think I misread your original comment. I thought you were asking if it works anywhere else besides GitHub Actions. I do have a doc explaining how to set up GitHub Actions: https://flakytest.dev/ci/ since yes, I think it's the most common option these days!
Congrats on launching flakytest.dev, Anže! The 'mute' feature is super clever; it lets teams ignore flaky tests without forgetting about them, which sounds like a lifesaver. Can't wait to see it grow and include more languages. Great job!
Report
Maker
@shoppietra Thank you Pan! I wanted to validate the idea first with Python since that's the language I'm most familiar with, but if people really like the idea I'll be adding more languages soon!
Congratulations on the launch of your product ! The 'mute' feature sounds like a game-changer for streamlining the testing process and maintaining control without unnecessary interruptions. It's great to hear about the simplicity of setup and the inclusion of essential features like statistics and error logs.
As you mentioned focusing on Python initially, I'm curious about your plans for expanding support to other languages and frameworks. How do you prioritize which languages and frameworks to add next based on user feedback and demand?
Report
Maker
@yasec_kowalski Thank you for the kind comment 🙏
Other languages and frameworks will be based on user feedback so do please let me know what combination you'd like to see added first! I'm thinking that JavaScript/Playwright might be a good next step to add since it's fairly popular combination and browser tests tend to be flakier than unit tests in general, but the final decision will be based on user demand!
Report
Congrats on the launch! FlakyTest.dev seems like a smart solution to a common problem. How does the 'mute' feature ensure that flaky tests are addressed in a timely manner without being overlooked or forgotten?
Report
Maker
@vasily_kolosov1 Thank you for the kind comment! 💚
The mute feature has a timeout, so if you don't fix the test in that timeout it is going to start failing again. My plan is to also implement reminder emails before the timeout runs out so that the person who muted the test and/or the author of the test are aware.
Brizy
DiffSense
DiffSense
Retail Connect
Intellectia.AI