How the Freeletics Team Goes From Hypothesis to A/B Testing

We review our progress and strategy each year at Freeletics. At the end of 2023, we wanted to grow three pillars: real-time coaching, selling more, and building a strong platform - making sure everything is iterable, scalable, and that we’ve paid down tech debt.

We took this whole strategy and broke it down into team goals. My team focused on real-time coaching and selling more, as that’s where we could add the most value. The third pillar is more ongoing work over the year but doesn’t require specific features to attack on the optimization side.

From there we decided how we would go about achieving these goals.

For our revenue goal, we saw the biggest opportunity was increasing revenue from new users who installed the app.

Some purchases are impulse conversions - they happen within one day. However, we also wanted to look at longtail conversions, too - those with usage of 30 days or more from installation.

Finally, we monitored basket size and how to increase it. But since Freeletics is a subscription business, basket size relates just to first monetization moment.

We weren’t sure where to start, so we started with three main steps.

The first step was to analyze the behavior of the user. That behavior could be something generic, for example, how users convert. We would look at how they behave before purchase, what actions they take, and how it affects their longevity.

Second was to talk to users—there’s no better way to know their needs and the reasons why they subscribe. Our analysis told us what was happening, but talking to users told us the why. It’s vital to get both perspectives.

You have to constantly work on customer feedback to build your backlog. No matter when you attack or start a certain feature, you always have this history of what people really need and where you can use those ideas to tackle the product strategy.

The third step was to enhance the user journey. We created a user journey map and put down all the channels where users came from to the app and website so you know what behaviors you want to analyze.

Analyzing user behavior involves doing all of these three things in parallel, enhancing them as we understood more about our users.

Amplitude is the easiest tool I’ve found for data analysis and doing things fast. We’ve used other things in the past, like SQL and other SQL-based chart tools. However, with Amplitude, if you know the events flowing into the infrastructure, you know how to create charts after a couple of tries.

When developing the roadmap, I started by creating a document in Confluence but it can just as easily be done using Amplitude’s notebook feature. I documented the status quo, stating how many people use Freeletics, what the conversion rate is within 30 days, how many converted within that time, and whether any converted within the first one or two days.

The idea was to lay down a user journey map right from the install until the user made the decision to purchase. You have different funnels that show you where the user enters, where in the journey they purchase, and where they drop off. This gave me a good idea of what the highest priorities were and some insights to work from.

For example, we could see that monetization was front-loaded. The median time was 28 minutes, meaning most monetization happened within the first few minutes of installing the app.

We also looked at when users returned to the app between days one to seven after installing and days eight to 30. We could see where people drop off and what we should improve.

Then we looked at where in the app users convert and which paywall they convert from. It was surprising to see that the order the paywalls perform is not the same in which the paywalls are structured as part of the funnel.

For example, let’s say there’s paywall A, B, and C. You might expect paywall A to do better than B in terms of purchases, but in some cases, the order shifted. One explanation could be that the user doesn’t understand the app at that point and instead buys it later. This can show an issue in the app design that we could work on as part of the strategy for impacting sales.

We had tried incentivizing users to purchase with a discount paywall, but we found that the majority of people, some 80-90%, didn’t even see the option as they closed the pop-up.