The What, When, and Why of User Retention

What percentage of users do you think come back to your app three days after downloading it? 90%? 60%? 40%?

Mobile intelligence company Quettra recently came up with a sobering statistic: The average Android app loses about 80% of its daily active users within the first three days, and about 90% by the first month. Think about it for a moment. 80% of the users who download your app are gone by the third day! It doesn’t even matter if you have thousands of people downloading your app. The harsh reality is, if they forget about it a couple days later, you’re deader than a Sean Bean character whose luck just ran out.

Fear not.

Instead of focusing on acquiring new users, you have to figure out how to hang on to the ones you have. You have to focus on user retention.

There are many ways to think about retention; what’s most lacking is a framework that lets you ask 1) what your retention is 2) where you have a problem and 3) how you can fix it. We like to call this the “3 levels of retention.” Each of these “levels” looks at user retention using a thicker lens and lets you ask a more complex question. Each question can be tackled by analytics tools of varying sophistication. In this post, we’ll start with the basic metric and then dive deeper until you’re able to ask key questions of your users’ behaviors; this will eventually allow you to develop better retention.

Level 1: What is my retention?

Say you’re a developer who’s thoroughly unsatisfied with all of the music streaming services on the market. You decide to create your own hipster music app called JukeBox. It’s been getting some traction, some users, but you know that if you want to get avoid a premature “Sean Bean”-type death, you need to know your retention.

The most superficial level of retention is simply getting at your numbers. There are different ways to calculate retention (which we’ll tackle in a later post), but for the sake of simplicity, let’s suppose you are looking at Day N retention. This is the most common type of retention to calculate. Day N retention reflects the percentage of users who come back and do any action in your app “N” days after they start using it.

If you don’t have analytics tools in place, you’ll have to calculate this metric by hand, manually leveraging software like Excel. This is a crude and inefficient way of doing things and can end up taking hours; most people can get a basic retention metrics from free analytics tools out there, but let’s treat this as as thought exercise.

Let’s say 12,481 users downloaded JukeBox on January 1, 2016 and you want to calculate the retention of this cohort, every single day, until Day 7. You need to track:

1. New users acquired on Jan 1 (This is Day 0)
2. Users from Jan 1 who were active on Day 1, Day 2, Day 3, … , Day 7

In this example, you start with 12,481 users on January 1. Let’s suppose you assign each user an ID in a spreadsheet and see that 3,506 of users who started on Jan 1 (i.e. D0) opened your app again on Jan 2 (i.e. D1). Your D1 retention rate for the Jan 1 cohort is: (3,506/12,481)*100 = 28.1%

If you repeat this process on Jan 3-7 (i.e. D2-D7), you may see something like this:

But drawing conclusions on this data would be imprudent; you’re only looking at one specific acquisition cohort – users who were acquired on Jan 1. Most out-of-the-box analytics tools can calculate the Day N retention of different daily, weekly, or monthly cohorts, within a customizable range of dates. For example, if you wanted to look at the retention of daily cohorts acquired within January 1-9, you’d get a table that looks something like this:

Great, so now you have a table full of numbers. If only you had some way of understanding what it all means! To get value out of your retention numbers, you have to go one level deeper.

Level 2: When are my users dropping off?

Retention curves are critical in pointing out when and where your app is having problems retaining users.

Okay, so now you have a retention curve which is visualizing the average Day N retention of your January acquisition cohorts. You see how the trend is going and you’re not happy. You’ve lost 60% of your users by Day 1; by Day 7, you’ve only held on to 14% of your users.

This is a problem. To stay in business, JukeBox needs to make revenue. To make revenue, you need users. But it’s 6-7 times more expensive to acquire a new user than to retain the ones you have; if there was a way to improve your user retention, it would be a lot more profitable for your company.

To do that, you need to understand why your users are churning.

Level 3: Why are my users not being retained?

About 25 years ago, a study published in the Harvard Business Review showed that increasing customer retention rates by 5% increases profits by 25% to 95%. Imagine if just a small tweak to JukeBox’s onboarding experience led to higher retention and more revenue.

To gain real insight into what could improve your retention numbers, you need to ask a key question about your users’ behaviors: What actions are correlated with users being retained? What actions reduce user churn?

To answer these questions, you need to utilize analytics tools that will let you compare the retention of various behavioral cohorts – that is, groups of users who perform a certain action.

As soon as users download JukeBox, there are a number of actions they can take. They can join a community, find friends, create a playlist, play a song, etc… More and more apps these days are incorporating elements of a social media network within them, so you hypothesize that encouraging users to join a community during JukeBox’s onboarding will be correlated with them being retained.

To test this, you first have to see how well the cohort of users who join a community are retained.

This retention graph shows that users who join at least one community are consistently retained better than the general pool of all users. Remember when we said the average app loses 80% of its users within the first three days? Well, only 28% of the users who joined communities churned out by Day 3. In fact, users in this cohort are consistently retained more than users who are not.

Okay, so what about the users who don’t join communities at all? If you plot the retention of users who do NOT join communities, you may get a curve like this:

Retention is especially bad for this cohort; 87% of users churn by Day 3!

What does all of this mean? If you’re a dev at JukeBox, it’s worth your time to figure out a way to entice users to join a community.

With some baseline understanding of your users and perhaps use of predictive analytics features, you can craft even more sophisticated behavioral hypotheses. Suppose JukeBox goes goes global – do users from Belgium in their mid-20s who join communities retain just as well as their American counterparts? If not, what behaviors do users in that specific demographic exhibit?

Conclusion

Retention is the king that Boromir could never be.

In this post, we’ve outlined three levels of complexity that make believe makes up a solid framework for acting on retention. Level 1 is your most basic question – what are the numbers? Level 2 is finding your problem areas. Finally, Level 3 is goes into the behavioral layer of figuring out why users are churning and how you can get them to retain.

We’ll be talking more about retention in the coming weeks. To learn more, subscribe to our blog and tweet us!

Comments

Yassin Shaar: Great share. Looking forward to the next ones.

Johan Arve: Assuming the app was downloaded at 10 PM the second day will commence two hours later according to the first method and at 10 PM on the following day according to the second method as the second 24 hour period would start first then.

Alicia Shiu: We calculate daily retention based on 24 hour periods — so in your example, the second day would start at 10 PM on the following day. You can read more about how we compute retention here: https://amplitude.zendesk.com/hc/en-us/articles/230543327