Hello Product Hunt. We built ABTestly because every testing tool we tried put a visual editor between the engineer and the page. That is fine for swapping a headline. It falls apart on a headless build, on a single page app, or anywhere the team has already refused to add a third party script that rewrites the DOM after it loads. ABTestly works the other way round. You write each variation as real JavaScript and CSS. It lives in your repository and goes through the same review as any other change. The runtime is about 31 KB gzipped, and the page is held until the variation is applied, so visitors do not see the control flash first. The part we care most about is the measurement. Assignment, exposure and conversion are recorded as three separate events on an exact ledger rather than one inferred number, so you can audit where a result came from. Confidence intervals are always shown with the sample size, no winner is declared before significance, and sample ratio mismatch is detected and flagged. Frequentist statistics are on every plan; sequential and Bayesian start at Pro. Pricing is published rather than quoted: $99 a month at 50,000 tracked users, $249 at 200,000 and $549 at 500,000. There is a 14 day trial on monthly plans and there is no free tier, so you will need a card to start. We are small and this is early. If you try it and something does not work, say so here and we will answer. The ABTestly team

ABTestly

Code-first A/B testing with statistics you can audit

Code-first A/B testing with statistics you can audit