3 Ways To Measure User Retention

This article is an excerpt from the first volume of The Product Analytics Playbook: Mastering Retention.

Customer retention is the one metric that matters for sustainable growth. The Playbook is a comprehensive guide to understanding user retention that provides a novel framework for analyzing retention at every stage of the user journey. You can find our first post from the Playbook here.

Retention impacts every important business metric that you (and your investors) care about—active user count, engagement, customer lifetime value, payback period, and more. A business that retains its users increases its revenue and becomes profitable faster than one that does not. And yet, keeping users engrossed in your app is no small task.

It’s especially daunting when you realize that you’re not just competing with other apps in your space—you’re competing with apathy. You’re competing with inattention. There are too many apps out there for anyone to wait patiently for yours to deliver value.

Even the best apps are going to lose the majority of their users in just a few days, but if you make user retention your team’s primary growth focus, you can totally change the trajectory of your company.

To cut through the static and distractions, you need to quickly deliver an experience that users want to return to again and again. There’s no better tool for doing so than analytics. As the old maxim goes, “what gets measured gets done.”

Let’s start with how we measure our retention.

3 Ways To Measure Your Retention

Retention is a measure of how many users return to your product over time.

What makes it confusing for many product managers and developers is that there are a few different ways to calculate retention, and it’s not always clear how your analytics platform is defining it.

When most people think about retention, they’re thinking about N day retention; at Amplitude we also like to think about (and allow you to calculate) what we call unbounded retention and custom bracket retention.

Figuring out which one makes the most sense for you is about understanding what you want to learn and how your users naturally behave. It’s unlikely that you’ll know this from the start, which is why most people start out with the most common definition of retention—N day retention.

1. N Day Retention

N day retention simply measures how many of your users come back to your app on a particular day.

You always start counting from “Day 0.” Day 0 could be the day that a user downloaded your app, the day they registered, or the day they performed any kind of action within your app—played their first song, added their first friend, etc. Then you can study that cohort over the days that follow, looking at how many returned (or performed another kind of action) on Day 1, Day 2, Day 7, and so on.

When you’re just beginning to probe your users’ behavior, N day retention will help you understand basic usage patterns. At this point, progress is all about making educated guesses about how users get value from your app and how well you’re delivering it. Looking at how many people came back to your app after one or two or three days isn’t the most elegant—but it will help you begin to test your assumptions about usage.

This retention curve shows the weighted averages of all of the Nth day retention numbers in a Day 0 to Day 30 timeframe.

In this example, 21.06% of users came back on Day 1 (beginning 24 hours after download) and performed some kind of action in the app, 16.82% came back on Day 2 (beginning 48 hours after download), and so on.

What do we know? Thanks to our research, we know that our Day 1 retention is right around average for an Android app. Also from that research, we know that bending that line between Day 0 and Day 1 up is one of the biggest opportunities we have for building retention.

N Day retention is well-suited for gaming apps, or any other type of app where you’re trying to get users to develop a daily habit. For social gaming app QuizUp, the most popular trivia game in the App Store, the most engaged user comes back day after day to challenge other players around the world. In fact, for QuizUp, completing a certain number of games within the first day is a key promoter of long-term retention.

Making your app more likely to retain users over that 24-48 hour period is crucial because these are when first impressions are made and solidified. No amount of “We miss you” emails are going to bring back someone who had a lousy first time with your app. Your N day retention gives you a solid basis for experimenting with, for instance, different kinds of onboarding flows, because at the conclusion of each experiment you can see whether first-time users are sticking around longer.

2. Unbounded Retention

Unbounded retention can be a much more meaningful way to measure retention if your product doesn’t rely on daily engagement. Unbounded retention shows you the percentage of users who came back on a specific day or anytime after that day.

Put another way, unbounded retention is actually the inverse of your churn rate. By measuring the inverse of your unbounded retention, you can see precisely how many users used your app on Day 0 and never returned again:

This is the same set of data-points that we looked at in the N day retention graph above. Notice our unbounded Day 1 retention rate is 50%, compared to only 21% for N day—that’s because half of all users who signed up on Day 0 came back at some point during the following month.

Unbounded retention analysis can be a better fit than N day retention for many types of businesses. For example, if you’re a grocery delivery service like Instacart, you definitely don’t expect people to use your product on a daily basis. It also doesn’t matter to you whether someone comes back exactly on Day 7 or Day 30, which is what N day retention gives you. Instead, it would be much more useful to look at your unbounded Day 7 retention, to see how many new users are returning to your app after the first week.

3. Bracket retention

Bracket retention is a bit of a catch-all, since you can basically define custom retention periods however you want to. Once you understand your users and their expected usage patterns, you can begin to use bracket retention. Bracket retention is a more nuanced version of N day retention that lets you split up your measurements into custom intervals of multiple days, weeks, or months. Here, four brackets are set up:

Day 0 (one day)
Day 1-3 (three days)
Day 4-6 (three days)
Day 7-11 (five days)

If any kind of activity in the app qualified a user as “active,” then someone would be counted as fully “retained” by:

Registering for the app on Day 0
Coming back on Day 1, Day 2, or Day 3
Coming back on Day 4, Day 5, or Day 6
Coming back on Day 7, Day 8, Day 9, Day 10, or Day 11

Here’s what a retention curve might look like with these brackets:

Each multiple-day bracket is like a bucket, and if a user is active anytime inside that bucket, they’re counted as retained. The spacing of each bucket in time is based on the pattern that we want to see our users exhibiting.

For an app people use every three weeks to stock up on home goods, we might create buckets that span about three weeks, plus or minus a week. In that case, we don’t care whether people come back exactly on Day 1 or Day 3, or whether their second order is after two weeks or a month—we just want to see a rhythm that proves they’re getting value.

Which retention is right for you?

Okay, so we’ve highlighted the key differences between N day, unbounded, and bracket retention. To summarize:

N day retention: The percentage of users that come back on a specific day.
Unbounded retention: The percentage of users who come back on a specific day or after.
Bracket retention: A more flexible version of N Day retention, where you can look at retention during custom timeframes.

How do you figure out which type of retention to use? There’s no quick and easy answer, but it depends on a combination of your product’s usage patterns and your business goals.

Figuring out which type of retention to use is heavily dependent on how frequently you expect people to use your app. If you expect people to come back on a regular basis, like daily for a mobile game, or weekly for an exercise app, then N day retention is probably a better fit. If you notice that many of your users don’t have a steady usage pattern — for example, a food delivery app where people place orders sporadically, then unbounded retention may provide a more accurate measure of how your business is doing.

To start out, you can try measuring your retention via a few different methods to see which one gives you the most meaningful information. The right retention metric should show you where you can improve and give you an accurate view of the health of your business.

If it’s not immediately clear what your app’s usage patterns are—not to worry. We have a framework you can follow to figure out just that. Refer to our Mastering Retention playbook to learn how to determine your app’s usage interval, how to set concrete retention goals, and when you should begin thinking about retention in the first place.