ABQ
Parallelize your test suite in CI through a message queue
5 followers
Parallelize your test suite in CI through a message queue
5 followers
ABQ runs test suites across parallel jobs in CI using a message queue. By using a queue, it ensures that your test suite always finishes as fast as possible. Integration takes minutes, with support for the most popular test frameworks.
👋 Hey, I'm Dan, cofounder of RWX. We're a team of six engineers that has been building dev tools for the past few years.
🚀 This is our first Product Hunt launch!
⚒️ We built a new parallel test runner called ABQ. It's for engineering teams that need to split their test suites across multiple jobs in CI.
🥉 The first step in that journey is usually to use round-robin partitioning and simply split the test files into two lists. Of course, that usually results in highly unbalanced partitions.
🥈 The next step is to try to use historical timings so that the partitions are expected to run as evenly as possible. It's a helpful evolution but usually has many shortcomings. Timings need to be maintained and updated. Also, high variance can result in the partitions still being unbalanced. And finally, partitioning based on timing assumes that all jobs start at the same time. If one CI job starts delayed, it will delay the entire run.
🥇Ultimately, we realized that the best way to distribute a test suite is the same way that an engineer would distribute any other workload for bulk processing – using a message queue. Load up all of the tests into a queue and let workers pull them off as fast as possible. It solves all of the problems with partitioning and ensures that the test suite will finish as fast as possible.
🤗 Once we built a generic test runner, we added some helpful features, such as the ability to automatically retry failed tests to help reduce the impact of flaky tests. We also added the ability to report on the test failures across all of the parallel nodes so that engineers only need to look in one place to see what failed across the entire run.
🆓 Since we initially launched ABQ we've separately built an entire CI/CD platform, and today we're excited to announce that we're making managed ABQ free for up to 4 parallel workers and 100 runs per month.
📖 ABQ is also open source, so you can run it yourself.
🤝 We'd love to help engineering teams that are looking for more optimal parallelization of their test suites in CI, so please get in touch if that's you!