Mastering Retention

Retention is critical for every product, whether you’re at a Fortune 500 company or a 5-person startup. Learn proven methods for building a data-informed retention strategy.

Table of Contents
                    CHAPTER 2

                    Your Critical Event & Product Usage Interval

                    Learn about which action in your product aligns with your core purpose, and why users are or aren’t responding to it.

                    This chapter introduces two important concepts:

                    1. What your critical event is
                    2. How often users naturally come back to use your product

                    In this chapter we will discuss

                    Before diving straight into our explanation of the Retention Lifecycle Framework, we have to explain two concepts related to your product’s usage: your critical event and usage interval.

                    As you will see in Chapter 3 and later, defining these concepts for your product is an important prerequisite to implementing the Retention Lifecycle Framework.

                    Before you get started

                    Before we dive into this chapter, take a moment to make sure you’re set up for success.

                    • Review instrumentation:
                      If you haven’t done so already, make sure your analytics is firing correctly and that you’re tracking the user actions that are important to you. Review your analytics instrumentation, organize the events you’re tracking, and validate your data before moving into the product usage sections of this chapter. Read more about how to conduct an instrumentation review in the Appendix.
                    • Measure baseline metrics:
                      Make sure you have a high-level understanding of your users and their behaviors before you put the Retention Lifecycle Framework in place. Get a baseline measure of your standard product usage metrics and make a note of it in the "Baseline Product Diagnostic" worksheet. You can then periodically track these metrics to see how your retention strategies change them. Take a look the Product Analysis Toolkit in Chapter 4 for the metrics and methods we recommend.

                    2.1 | Understanding critical events

                    Your product’s critical event and usage interval will inform how you carry out further analyses using the Retention Lifecycle Framework, so it’s incredibly important to think through how they apply to you.

                    What is a critical event?

                    A critical event is an action that users take within your product that aligns closely with your core value proposition. Chances are you probably already know what your critical event is—it’s the action that you want to drive your users toward and get them to perform.

                    Put another way, when you’re measuring retention, the critical event describes the action you want users to perform in order to be counted as truly active or retained.

                    2.1.1 | How many critical events can you have?

                    A good rule of thumb is to have one critical event per core product offering. For most companies this means just one.

                    In some rare cases, it may be appropriate to have multiple critical events. The most common example of this is two-sided marketplaces, which have two distinct product flows—like buying and selling, for example. These products’ users tend to fall into one group or the other, so it makes sense to have two critical events and analyze these users separately.

                    Uber, for example, has a user base of drivers and riders; Airbnb has hosts and guests; Etsy has buyers and sellers.

                    2.1.2 | Examples of critical events

                    Example 1: Airbnb’s Critical Event

                    If you were measuring Airbnb’s retention numbers, would you really want to count a user as retained if all they do is open up the app and browse listings? Simply opening up the app doesn’t provide any business value to Airbnb, nor does it align with their objective of generating revenue.

                    Airbnb’s critical event is making a booking. The company’s growth and success depends on hosts listing accommodations on Airbnb and users booking them.

                    Example 2: Our customers

                    Here are a few more examples of critical events from our customers. Note that in each instance, the critical event is closely aligned with the core value that the business provides its users.

                    Recap: Determining your critical event

                    In order to determine your critical event, here are some things you should ask yourself:

                    • What is the one action that you want a user to do every time they use your product?
                    • What metrics do you care about as a company? What number are you ultimately trying to drive up? Which user actions can be tied to that metric?
                    • Do you have different product offerings? If so, what are they? What are your success metrics for each?

                    2.2 | Determining your product’s usage interval

                    You can’t draw conclusions about your retention numbers without first having an understanding of your product’s usage interval. Some products are built to be used daily—think social networking, media, casual gaming, or productivity apps. Others, like on-demand, e-commerce, and expense reporting apps, would be used much less frequently.

                    2.2.1 | Why does the product usage interval matter for retention?

                    Say you have an on-demand restaurant delivery app. Your product intuition tells you that most of your users place an order about once a week. If you look at the proportion of users who come back on a daily basis, you might see something like this:

                    Comparing Daily and Weekly Retention

                    A massive drop-off after Day 0, and then a scarily low proportion of users coming back on any subsequent day. But that retention curve isn’t a good indication of the health of your product; N-Day retention would only give you the proportion of users who are active on one arbitrary day. These numbers will be small because most of your users won’t place an order every single day.

                    Instead, you should be looking at your retention on a week by week basis. That is, the proportion of your users come back any time during Day 1-7, Day 8-14, Day 15-21, etc.

                    Compared to daily retention, weekly retention gives you a much better indicator of your product’s health because it’s aligned with how frequently your users naturally come back to your product.

                    TERMS TO KNOW:

                    The product usage interval is the frequency (daily, weekly, monthly, etc.) with which you expect people to use your product.

                    To be able to accurately calculate user retention across all stages of the Retention Lifecycle (more on this in Chapter 3), you have to first determine how often you can expect users to come back to your product. Not doing so can cause you to misinterpret your retention metric and misinform your strategies to improve it.

                    2.2.2 | The Usage Interval Framework

                    So how do you figure out what your app’s usage interval is? You might already have some idea based off your own product intuition. If you don’t, not to worry. This four-step framework utilizes your existing user behavior data to help you determine your product’s usage interval with certainty.

                    1. Identify all users who repeated the critical event at least twice within a certain time period. We suggest 60 days. Note: you’ll want to use a time period that is longer than your usage interval — for most products 60-90 days is sufficient, since usage intervals rarely go beyond one month. This means we expect users to perform the critical event at least twice within 60 days.
                    2. Analyze how long users in step 1 took to come back and do the critical event for the second time.
                    3. Plot the proportion of users who repeated the critical event over different time intervals. This will give you a cumulative distribution function.
                    4. Identify the time interval at which 80% of users have repeated the critical event. This is your product usage interval.

                    Let’s put this framework in context by looking at a customer example.

                    2.2.3 | Determining your usage interval in Amplitude

                    The steps we outlined on the previous page will help you calculate your usage interval, no matter what analytics platform you’re using. Just follow the instructions in the worksheet at the end of this chapter to walk through those steps.

                    If you’re using Amplitude, however, you can use the Usage Interval view in the Retention Analysis chart to quickly find your product’s usage interval.

                    Let’s look at an example from one of our customers—a mindfulness app. Since this app’s core value is in guiding its users through mindfulness exercises and meditation, their critical event is completing a meditation session.

                    This team wanted to understand how often their current users return to meditate. In a retention chart, we set both the first event and the return event to ‘Meditation session completed.’

                    Retention Analysis on a Meditation Session

                    Then, click on ‘Change to Usage Interval View.’ The resulting curve shows the distribution of how long it took people in the Current Users cohort to repeat the critical event of ‘Meditation session completed.’ By locating the inflection point of this curve, you can approximate your product’s usage interval.

                    For example, the inflection point for this chart happens around 7 days. The chart shows that 62% of current users have repeated the critical event of completing a meditation within 7 days. We can say that users of this app can be expected to come back about once a week to meditate, or that this app is a weekly usage product.

                    Retention Analysis | Meditation Session Measurement
                    PRO TIP:

                    Excepting rare use cases like tax prep software, most businesses won’t have longer than monthly usage intervals. If you find your product to be an exception, feel free to adapt this framework to your needs.

                    Determining your product usage interval is a critical step in getting an accurate baseline reading of your current userbase, as well as a starting point for analyzing user retention according to the Retention Lifecycle Framework.

                    Complete the worksheet at the end of this chapter to determine your product’s usage interval using this framework.

                    One more thing…

                    As we said before, determining your product’s usage interval should inform your retention analysis and your retention strategies.

                    This framework is great for determining your product usage in a methodical, quantitative way (which is how we like to do things at Amplitude!). But it’s also important to supplement this with your own product intuition, as well as with solid user research. Qualitative user feedback can be just as valuable as, if not more than, your quantitative findings.

                    2.3 | Take action

                    This chapter was all about laying the proper groundwork for tackling the more in-depth retention analyses that will be presented in the rest of this playbook. Take some time to complete the following action items from this chapter. This will ensure your success in the future chapters.

                    • Check your analytics instrumentation
                    • Organize your event taxonomy
                    • Determine your critical event(s)
                    • Complete the worksheet "Determining Your Product Usage Interval"
                    • Complete the worksheet "Baseline Product Diagnostic"

                    2.4 | Product diagnostic worksheets

                    Mastering retention metric table
                    Product Usage Interval