How we used Truflag to increase our app's revenue by 33% in our proof of concept

One of the best use cases for feature flags is testing monetization without shipping custom logic to every user cohort. I recently used Truflag to run a pricing experiment inside my NFL trivia app and the results were bigger than I expected:

9 months ago I shipped an NFL trivia app (Gridiron Trivia) and began monetizing it with in-app subscriptions/purchases in July. The business model is pretty similar to New York Times games: offer daily games for free (with ads) and offer collections of games for one-time purchase. Subscribers get the ad-free version and access to all game collections.

The app got to 10K MAU by the end of the NFL season (no ad spend) and now natural growth has started to plateau since the season ended. I decided the best usage of time was to attempt to optimize the in-app purchase strategy to increase LTV, so I ran an experiment for the past 2 months and thought I’d share the results here as my changes ended up making a pretty big difference.

Prior to the experiment, our subscription pricing and basic statistics were as follows:

• Subscription Pricing: $4.99/mo | $39.99/yr | $59.99 lifetime

• Split: 80% | 15% | 5%

• Conversion to Paid: 5.3%

• Relative LTV: lifetime LTV > yearly LTV >>> monthly LTV.

The goal of the experiment was to test whether providing special pricing to user cohorts increases LTV over the status quo.

Experiment Design: New users were randomly assigned to 1 of 4 groups with equal distribution:

• Group A: $24.99/yr special offer

• Group B: $34.99 lifetime special offer

• Group C: 7-day free trial then $4.99/mo

• Group D: Control

I used Truflag to manage the experiment:

• Created a flag that served either A, B, C, or D

• Created a metric to track user purchases and subscriptions

• Created an experiment with the flag and metric for 100% of new users with 25/25/25/25 distribution

• Read the flag using the SDK and pop a modal (after 3 games played) with the special offer (or control)

Experiment Results:

All 3 groups outperformed the control by over 25% from the LTV perspective*, with a dip in initial user revenue with Group C. The primary metric we cared about was the expected LTV impact, and Group A’s +33.0% improvement over control was the highest.

This ended up being a great example of why we built Truflag in the first place: simple cohort-based experiments, controlled rollout, and clean measurement without a bunch of custom plumbing.

*LTV calculations were based on observed user churn for monthly and expected user churn for yearly.