Flagsmith Experiments - Open-source A/B testing built on your feature flags

Hi Product Hunt, I'm Asaph, Head of Product at Flagsmith. Product and Engineering teams have been using Flagsmith flags for years to release safely. Plenty of them already split traffic with our multivariate flags, and then try to track and join exposure and conversion events using other tools to see whether the change worked. Many teams stop before getting any results, or find the process so complicated that they never try again. So we built the other half into Flagsmith - Today we’re launching Flagsmith Experiments as a public beta, in the open-source core and on Flagsmith Cloud. How it works: - Start from an A/B flag you already run, in your code, using our SDKs - Connect your warehouse - we currently support ClickHouse and Databricks, and BigQuery and Snowflake will be available soon. On our Flagsmith Cloud you can use a fully managed warehouse with a 2-click setup. - Create an Experiment with a hypothesis, and pick the audience using Flagsmith Segments or from external cohorts we sync with (we support Amplitude and Mixpanel) - Create a metric from your events, or use an existing one from your warehouse. - Get results based on a Bayesian engine, with Chance to beat and Lift indicators. - End the experiment and you're back on the flag - roll out the winner or roll back Where the data lives is your call. Bring-your-own-warehouse is part of the open-source edition. We kept v1 small on purpose: a Bayesian engine, one primary metric per experiment, and SDKs for JavaScript, React, Python, Java, Node.js, Go and Flutter. The fastest way to get started is to use our prebuilt prompt with your favorite coding agent, visit our docs for more info. One thing that surprised us while dogfooding this ourselves is that our experiments gave us better questions more often than final answers. That’s why we designed it to show the result, and a possible recommendation, but the call is still yours. What would you need to run your next experiment here? Gagan, Wadii, both Matts and I are here to discuss and answer your questions (while we’re actually together at our engineering offsite in Portugal!). And to help you get started we’re offering the first 20 teams a free 30-min setup session with me - to claim your session let me know in the comments what you’d test first!