Mastering Retention

Before diving straight into our explanation of the Retention Lifecycle Framework, we have to explain two concepts related to your product’s usage: your critical event and usage interval.

As you will see in Chapter 3 and later, defining these concepts for your product is an important prerequisite to implementing the Retention Lifecycle Framework.

Before you get started

Before we dive into this chapter, take a moment to make sure you’re set up for success.

• Review instrumentation:
  If you haven’t done so already, make sure your analytics is firing correctly and that you’re tracking the user actions that are important to you. Review your analytics instrumentation, organize the events you’re tracking, and validate your data before moving into the product usage sections of this chapter. Read more about how to conduct an instrumentation review in the Appendix.
• Measure baseline metrics:
  Make sure you have a high-level understanding of your users and their behaviors before you put the Retention Lifecycle Framework in place. Get a baseline measure of your standard product usage metrics and make a note of it in the "Baseline Product Diagnostic" worksheet. You can then periodically track these metrics to see how your retention strategies change them. Take a look the Product Analysis Toolkit in Chapter 4 for the metrics and methods we recommend.

Your product’s critical event and usage interval will inform how you carry out further analyses using the Retention Lifecycle Framework, so it’s incredibly important to think through how they apply to you.

What is a critical event?

A critical event is an action that users take within your product that aligns closely with your core value proposition. Chances are you probably already know what your critical event is—it’s the action that you want to drive your users toward and get them to perform.

Put another way, when you’re measuring retention, the critical event describes the action you want users to perform in order to be counted as truly active or retained.

A good rule of thumb is to have one critical event per core product offering. For most companies this means just one.

In some rare cases, it may be appropriate to have multiple critical events. The most common example of this is two-sided marketplaces, which have two distinct product flows—like buying and selling, for example. These products’ users tend to fall into one group or the other, so it makes sense to have two critical events and analyze these users separately.

Uber, for example, has a user base of drivers and riders; Airbnb has hosts and guests; Etsy has buyers and sellers.

Example 1: Airbnb’s Critical Event

If you were measuring Airbnb’s retention numbers, would you really want to count a user as retained if all they do is open up the app and browse listings? Simply opening up the app doesn’t provide any business value to Airbnb, nor does it align with their objective of generating revenue.

Airbnb’s critical event is making a booking. The company’s growth and success depends on hosts listing accommodations on Airbnb and users booking them.

Example 2: Our customers

Here are a few more examples of critical events from our customers. Note that in each instance, the critical event is closely aligned with the core value that the business provides its users.

Recap: Determining your critical event

In order to determine your critical event, here are some things you should ask yourself:

• What is the one action that you want a user to do every time they use your product?
• What metrics do you care about as a company? What number are you ultimately trying to drive up? Which user actions can be tied to that metric?
• Do you have different product offerings? If so, what are they? What are your success metrics for each?

You can’t draw conclusions about your retention numbers without first having an understanding of your product’s usage interval. Some products are built to be used daily—think social networking, media, casual gaming, or productivity apps. Others, like on-demand, e-commerce, and expense reporting apps, would be used much less frequently.

Say you have an on-demand restaurant delivery app. Your product intuition tells you that most of your users place an order about once a week. If you look at the proportion of users who come back on a daily basis, you might see something like this:

A massive drop-off after Day 0, and then a scarily low proportion of users coming back on any subsequent day. But that retention curve isn’t a good indication of the health of your product; N-Day retention would only give you the proportion of users who are active on one arbitrary day. These numbers will be small because most of your users won’t place an order every single day.

Instead, you should be looking at your retention on a week by week basis. That is, the proportion of your users come back any time during Day 1-7, Day 8-14, Day 15-21, etc.

Compared to daily retention, weekly retention gives you a much better indicator of your product’s health because it’s aligned with how frequently your users naturally come back to your product.

TERMS TO KNOW:



The product usage interval is the frequency (daily, weekly, monthly, etc.) with which you expect people to use your product.

To be able to accurately calculate user retention across all stages of the Retention Lifecycle (more on this in Chapter 3), you have to first determine how often you can expect users to come back to your product. Not doing so can cause you to misinterpret your retention metric and misinform your strategies to improve it.

So how do you figure out what your app’s usage interval is? You might already have some idea based off your own product intuition. If you don’t, not to worry. This four-step framework utilizes your existing user behavior data to help you determine your product’s usage interval with certainty.

1. Identify all users who repeated the critical event at least twice within a certain time period. We suggest 60 days. Note: you’ll want to use a time period that is longer than your usage interval — for most products 60-90 days is sufficient, since usage intervals rarely go beyond one month. This means we expect users to perform the critical event at least twice within 60 days.
2. Analyze how long users in step 1 took to come back and do the critical event for the second time.
3. Plot the proportion of users who repeated the critical event over different time intervals. This will give you a cumulative distribution function.
4. Identify the time interval at which 80% of users have repeated the critical event. This is your product usage interval.

Let’s put this framework in context by looking at a customer example.

The steps we outlined on the previous page will help you calculate your usage interval, no matter what analytics platform you’re using. Just follow the instructions in the worksheet at the end of this chapter to walk through those steps.

If you’re using Amplitude, however, you can use the Usage Interval view in the Retention Analysis chart to quickly find your product’s usage interval.

Let’s look at an example from one of our customers—a mindfulness app. Since this app’s core value is in guiding its users through mindfulness exercises and meditation, their critical event is completing a meditation session.

This team wanted to understand how often their current users return to meditate. In a retention chart, we set both the first event and the return event to ‘Meditation session completed.’

Then, click on ‘Change to Usage Interval View.’ The resulting curve shows the distribution of how long it took people in the Current Users cohort to repeat the critical event of ‘Meditation session completed.’ By locating the inflection point of this curve, you can approximate your product’s usage interval.

For example, the inflection point for this chart happens around 7 days. The chart shows that 62% of current users have repeated the critical event of completing a meditation within 7 days. We can say that users of this app can be expected to come back about once a week to meditate, or that this app is a weekly usage product.

PRO TIP:



Excepting rare use cases like tax prep software, most businesses won’t have longer than monthly usage intervals. If you find your product to be an exception, feel free to adapt this framework to your needs.

Determining your product usage interval is a critical step in getting an accurate baseline reading of your current userbase, as well as a starting point for analyzing user retention according to the Retention Lifecycle Framework.

Complete the worksheet at the end of this chapter to determine your product’s usage interval using this framework.

One more thing…

As we said before, determining your product’s usage interval should inform your retention analysis and your retention strategies.

This framework is great for determining your product usage in a methodical, quantitative way (which is how we like to do things at Amplitude!). But it’s also important to supplement this with your own product intuition, as well as with solid user research. Qualitative user feedback can be just as valuable as, if not more than, your quantitative findings.

This chapter was all about laying the proper groundwork for tackling the more in-depth retention analyses that will be presented in the rest of this playbook. Take some time to complete the following action items from this chapter. This will ensure your success in the future chapters.

• Check your analytics instrumentation
• Organize your event taxonomy
• Determine your critical event(s)
• Complete the worksheet "Determining Your Product Usage Interval"
• Complete the worksheet "Baseline Product Diagnostic"

• The Amplitude Guide to Customer Retention: 40+ Resources to Increase Retention, The Amplitude Team
• Customer Retention 101, The Amplitude Team
• Why Retention is the King of Growth Strategy, Brian Balfour, Co-founder, Reforge & former VP of Growth, Hubspot
• New data shows losing 80% of mobile users is normal, and why the best apps do better, Andrew Chen
• How To Increase Growth Through Retention Analysis, Archana Madhavan, Amplitude Blog
• 6 Worksheets to Understand, Improve, and Calculate Retention Rate