Mastering Retention

Remember, a resurrected user is someone who is active in the current period, was not active in the previous period, and was active at some point before that.

Just like you did for new and current users, take your resurrected user cohort that you created in Section 3.3 and plot your baseline retention for resurrected users.

It’s helpful to look at longer-term effects, at least 1-2x your product’s usage interval out. Here’s the retention curve of resurrected users for one of our customers, an on-demand delivery company.

How do resurrected users retain compared to other users?

You should also compare this with the retention curves of your current users and new users. This will give you a sense of how your resurrected users currently perform relative to these other two groups.

Example: Resurrected users and events

In the chart, we’ve added the retention curves of current and new users for the on-demand delivery company. While resurrected users don’t retain as well as current users, they do retain better than new users during the same time period.

You can see that resurrected users have significantly better rates than new users during the same time period of returning and placing an order, even many weeks after they initially resurrect.

Takeaway
In this case, resurrected users already retain and place orders at much better rates than new users. This indicates that resurrecting users could be a good source for gaining more current users and increasing revenue.

Here’s a different situation for a product where the retention for resurrected users is very low, even lower than for new users. This could mean that the product isn’t doing a good job of showing value to resurrected users, so people are not reengaging and end up just leaving the app.

Takeaway
The good news is that this product has strong current user retention. For a situation like this, we recommend looking at what it would take to increase resurrected user retention, and weighing that effort against the potential benefit. For smaller teams with limited time and resources, it might make more sense to focus on improving new user retention in the short term.

Now that you have your baseline resurrected user retention and know how these users retain relative to current and new users, it’s time to assess whether resurrected users might be a good source of growth for your product. Two calculations that will help you determine the opportunity size are:

• The percentage of your active users that are resurrected
• How many potential resurrected users you have

Calculate the breakdown of your active users during the current time period you’re measuring. In Amplitude, you can do this using the Lifecycle feature (see Section 3.3). Here’s a hypothetical example of active users during a week:

The first thing to note is that 73% of the active users during this week are current users—people who have been using the app with some consistency. That’s great—it means this product has a healthy base of users, and isn’t just pouring on new users who quickly churn.

Notice that resurrected users actually make up 14% of the active users, which is about equal to the number of new users during that week. This indicates that the company is already successfully resurrecting users (whether organically or with targeted marketing efforts), and increasing their efforts here could have a positive impact on overall retention.

Next, let’s look at the size of the potential pool of resurrected users. Anyone who has used your product in the past but has not used it in the current period of analysis is a potential resurrected user. A practical way to assess the size of this opportunity is to calculate the number of people who used your product sometime in the preceding 6 months, but have not used it in the current period. Depending on the type of product you have, your usage interval, and any seasonality of your product, you may want to look at a period that’s longer or shorter than 6 months, but it’s a good place to start.

In Amplitude, you can calculate this group of users with a behavioral cohort definition. If you’re looking at the current time period of the week of July 3 – 9, you would create a cohort of users who were active at any time in the last 6 months, but were not active in the current week:

Going back to our example, this product has 1.3 million potential resurrected users. Think about this from an acquisition perspective. This company has 1.3 million people who have downloaded their app in the past 6 months, but are currently inactive. These are people they can try to reengage with a well-timed push notification or email, and are much easier to reach than all of the potential new users that they’re spending money to acquire.

Just like you did for new and current users, you should examine behavioral personas of your resurrected users. Understanding these patterns of behavior can show you why users might be returning, or what may have triggered their resurrection. For a refresher on behavioral personas, see Chapter 4.

Example: Resurrected personas for on-demand delivery

An on-demand delivery used Amplitude Personas to find clusters of users within their resurrected user cohort. They identified a few interesting personas, listed in the table.

The first 2 personas are especially encouraging—people in these personas are placing an order when they return to the app, plus almost all of them are retained 2 months later. These personas, who together make up 20% of resurrected users, are using the product as expected when they return.

The other 2 personas, ‘Just browsers’ and ‘Discount redeemers’, provide some interesting information.

Browsers

The ‘Just browsers’ persona contains a lot of users, making up 21% of the resurrected user cohort. These people exhibit browsing behavior on a similar level to ‘Browsers who order’, but then ultimately don’t complete an order that day. As a group, they have pretty high 2 month retention at 74.52%, showing that even though they don’t complete an order on that day, there’s a high likelihood they’ll come back later.

Takeaway
Improving the browsing experience could be an effective way to boost resurrected user retention, since 21% of resurrected users are browsing but ultimately not ordering. If the company could get more ‘Just browsers’ to become ‘Browsers who order’, not only would they get more revenue from orders on the day they resurrect, but this data shows that ‘Browsers who order’ have much higher long-term retention as well (97% 2 month retention).

Discount redeemers

The ‘Discount redeemers’ Persona showed several events related to redeeming a discount on their next order, which was sent to them via email.

Any time you’re looking at the impact of discounts, you need to measure how well they incentivize users to place an order not only in the short-term, but as a repeat customer moving forward. We’ll look at that more in the next section.

PRO TIP:
In the book "Hooked," author Nir Eyal talks about two types of triggers: external and internal.

External triggers are things like push notifications, emails, or ads that we use to get users’ or potential users’ attention. Many mobile apps use push notifications to encourage users to come back to their app.

Internal triggers, on the other hand, happen in a person’s mind. According to Nir, an internal trigger occurs when "a product is tightly coupled with a thought, an emotion, or a pre-existing habit." For example, we open Facebook when we’re feeling bored or lonely—the impulse to open Facebook is cued by emotions.

The best habit-forming products start out with external triggers to initially attract and educate the user, but over time users no longer need the external triggers to keep using the product, relying instead on internal ones.

The next step is to determine any measurable triggers of resurrection. These may be push notifications or emails that you send to your users—for example, if you already have some reengagement campaigns targeted at users who have been inactive for some amount of time. If your product has a social component, these notifications could be based on actions from users’ friends or networks (like the notification you get when someone mentions you on Twitter).

Your product might also have triggers that coincide with outside factors like holidays, sporting events, or weather. For example, you might notice more users placing orders with an on-demand app during a week of heavy snow, when people are less inclined to go outside to run the errands they normally would. It’s hard to confirm these factors in your data, but anytime you notice spikes or dips in usage, don’t forget to think about these outside influences on your users.

For resurrected users, we want to identify any external triggers that might have brought them back to the product. Once we identify them, we can measure how effective they are at reengaging users and think of ways to improve.

Here are a few ways to identify triggers

• If you have a website or web app, look at session utm parameters and referrer data to look for common sources, like an email campaign or ad.
• If you do a cluster analysis to determine behavioral personas, like we did in the previous section with the Personas feature, you can look for events within each cluster that could have triggered resurrection.
• Analyzing the paths of resurrected users, using Amplitude’s Pathfinder or a similar path visualization, can also shed light on why users are returning.

Once you identify potential triggers, compare downstream metrics like the critical funnel conversion rates and long-term retention for users who receive these triggers. This will allow you to measure whether they’re having the intended effect.

PRO TIP:
Tracking messaging data (push notifications and emails) and attribution data in your product analytics platform enables you to measure the impact of these campaigns on later in-product behavior.

We recommend sending messaging and attribution data to your product analytics so that you can get the full picture of user behavior.

If you’re using Amplitude, we partner with best-in-class providers across messaging and attribution so that you can easily integrate different data sources into Amplitude.

Example: The impact of discount email offers

In the previous section, we discussed the ‘Discount redeemer’ persona for the on-demand company. These discounts are sent via email to a subset of users who have been inactive (not completed an order) for some amount of time. We created a behavioral cohort of resurrected users who had received the special offer, and found that 22% of all resurrected users had received a discount offer.

When we compared the critical funnel conversion rates for resurrected users who received the special offer, versus those who did not, we found a huge difference—94.5% of resurrected users who received a special offer completed an order, compared to only 26.5% for the rest of the resurrected users.

Basically, almost every single user who came back to the app after receiving a discount completed an order. The team also looked at the conversion rate from receiving a discount to launching the app, and found that 58% of users who receive a discount offer go on to launch the app within 7 days.

Now that the company knew that the special offer was effective at getting people to come back and place an order, they wanted to look at the long-term impact. Are users just coming back the one time with the special offer, or do they keep placing orders over time?

Looking at monthly retention moving forward, where the returning event is to ‘Complete order’, you can see that people who got the special offer retain at significantly better rates than those who did not — even many months later.

Takeaway
Clearly, the discount program has a significant long-term effect on increasing purchases. The on-demand company decided to try expanding this program to more of their dormant users to encourage resurrection.

Notification spam doesn’t work!

It might be tempting to blast your dormant users with notifications or emails, but chances are these will not be effective and will only serve to annoy your users, causing them to unsubscribe or worse, stop using your product forever.

Remember that external triggers, like push notifications, need to be well timed with a user’s internal triggers and existing behavior. Notifications work best when they redirect existing emotions or behaviors to your product. They’re even better when you can personalize them based on something you know about the user—whether it’s preferences they’ve set or prior actions they’ve taken.

Once you identify some triggers and behavioral personas, go back and compare these to your current and new users. We recommend looking at long-term retention and critical funnel conversion rates. You can also measure revenue and use any other methods from the Product Analysis Toolkit in Chapter 4 to help you assess how valuable current users are to focus on relative to other groups.

Example: Compare long-term retention & critical funnel conversion rates

Continuing the on-demand company example from the previous sections, the company split resurrected users into 2 main personas: Organic (did not receive any email reactivation campaign) and Non-Organic (did receive a reactivation campaign). Then, they compared the retention of these two resurrected user personas to current and new user retention.

Here is the weekly retention chart, where the returning event is placing an order (the critical event). Organic resurrected users have retention that is about 65% higher than for new users, while non-organic resurrected users retain far better than new users—they have 260% greater retention. These retention impacts are also long-term, extending 24 weeks out.

Compare critical funnel conversion rates

Comparing the critical funnel conversion rate for resurrected users compared to new and current users will show you whether resurrected users are currently helping your business goals. You can also identify any critical drop-off points for resurrected users, and see what dropped-off users do instead of converting.

Here’s the critical funnel, comparing the same user groups during the current period. The funnel shows the conversion rate from opening the app to completing an order. Non-organic resurrected users have the highest conversion rate, at 94.6%. Organic resurrected users have a slightly lower conversion rate than new users: 28.2% compared to 33.2%.

Takeaway
For this company, reactivation email campaigns not only increase conversions during the current period, they also have a significant positive impact on retention and repeat orders over at least the next 24 weeks. This is a really good indication that their campaigns are working, and they should try sending campaigns to more of their dormant users.

In addition, this data shows that resurrected users as a whole have higher conversion rates and place more orders in the long-term than new users, especially non-organic resurrected users.

Measure critical events and event properties

Remember, your critical event is the user action that represents that a user is actually using your product and getting value out of it (e.g. completing a game, placing an order, playing a song). When analyzing your resurrected users, make sure you investigate how much they perform your critical event, not just whether they return to the product.

Comparing critical events and event properties helps you investigate whether resurrected users have different behavioral patterns or perform these events at a different frequency than current users. This can help you determine whether:

• It’s worth focusing your efforts on resurrecting more dormant users, from a retention or monetization perspective
• There’s something unique to how resurrected users behave that you should use to tailor their experience upon return

Compare engagement for critical events

One way to compare critical event engagement between different user cohorts is to measure the percentage of users in each cohort that did the event. Remember to also graph any significant personas of resurrected users that you’ve identified.

Example: Critical events for a lifestyle product

Our lifestyle customer’s critical event is booking an appointment. In the chart below, they looked at this critical event as a percentage of active users in each cohort. In other words, the graph is showing what percentage of each cohort (current, new, or resurrected) that booked an appointment on each day.

As you can see, resurrected users have a lower percentage of their users booking an appointment when compared to new and current users.

Another way to look at engagement is the average number of times a user does the event. In the next graph, we see a similar pattern, that resurrected users on average book fewer classes than new and current users.

Compare important event properties

While the number of bookings that users make is an important metric for this company, the end goal is revenue — how much users actually spend. So, they looked at ‘cart total price’, which is an event property for their booking event.

Graphing the average cart total price for each group of users, they found that resurrected users are actually spending more than new users on average for each transaction.

Takeaway
Even though the total number of booking is less for resurrected users, they spend more per transaction and there is a large pool of potential resurrected users for this company. We recommend dedicating some time and resources to resurrect users.

Compare stickiness and session metrics

In addition to looking at critical event patterns and event properties, you can look at stickiness and session metrics to get at other aspects of resurrected user engagement. We covered these metrics in Chapter 4, so feel free to go back and review those sections for more details on how to measure them.

Stickiness and session metrics are another way to compare the engagement of your resurrected users to the behavior of current users. By uncovering any differences, you can form hypotheses about why resurrected users are different and find ways to get resurrected users to behave more like current ones.

We recommend doing the following analyses

• Compare stickiness of your critical event(s) for resurrected, new, and current users
• If session length is important for your business, you can compare session length distributions and average session length for resurrected, new, and current users

Session Metrics

If session length is a good indicator for engagement for your product, try graphing the distribution of session lengths for resurrected users and compare that to the usage of current users. Similar distributions will indicate that resurrected users behave similarly to current users, so it should be easier to get them to become current users in the long-term.

Example: Session length as an indicator for engagement

For the mindfulness app that we’ve discussed, session length is a good indicator for engagement. The more time a user spends in the app, the more value they’re getting from the product.

Looking at the average session length, the team found that current users on average have longer sessions that resurrected users.

When looking at the distribution of session lengths, they found that resurrected users have a much higher percentage of sessions that are only less than 30 seconds long. For this product, there’s not much a user can accomplish in less than 30 seconds, so we can assume those users aren’t really using the app during that time.

Current users also have a much higher proportion of sessions that last 10 minutes or longer.

Takeaway
This data shows that current users spend more time in the app and have longer, likely more meaningful sessions than resurrected users. If this company wants to reengage resurrected users, they likely need to improve the resurrected user experience in order to encourage them to behave more like current users.

Compare revenue for resurrected, current, and new users

Of course, the bottom line for most businesses is revenue. Comparing the revenue for resurrected users (and any important personas) relative to current and new users will help you decide whether resurrecting users is a worthwhile pursuit for your team.

As we mentioned before, it will likely cost you far less to resurrect a user than to acquire a new one. So, by comparing the potential monetization of resurrected users compared to new, you can determine the relative ROI and decide how you want to spend your resources.

We recommend comparing revenue metrics like:

• ARPU: Average revenue per user
• ARPPU: Average revenue per paying user

Example of ARP

In the graph to the right of ARPU, resurrected users spend more per user than new users during the same timeframe, and spend about the same amount as current users.

The ARPPU values are closer together, but still shows that per paying user, resurrected users are spending more than new users and are spending similar amounts to current users.

Remember, the overall goal of resurrected user retention analysis is to learn how you can "resurrect" or reactivate dormant users and get them to become current users of your product. You also want to get a sense of the potential value of resurrected users and whether you should spend your efforts resurrecting more users, especially as compared to your resources spent acquiring new ones.

Although not always the case, we’ve seen that for several of our companies, resurrected users convert and retain better than new users, as well as contribute more revenue per user. And since you’ve already acquired those users, the cost to resurrect a user through a push notification, email, or special offer is likely to be less than the cost of acquiring a new user.

You’ll need to do your own analysis to make sure this is true for your business, but it’s certainly worth investigating as an (often overlooked) source of growth.

Key questions to ask yourself as you form hypotheses:

• What percentage of your active users currently come from resurrected users, and what’s the potential active user growth you could get from increasing the number of resurrected users?
• Are there any ways you can tailor the experience for resurrected users to increase the chance that they re-engage and become a current user?
• Is there an opportunity to trigger more users to resurrect that will provide an overall lift in your retention and other core metrics?
• What are effective triggers that you can experiment with for resurrecting dormant users?

You can use the Resurrected User Retention worksheet at the end of this chapter to take notes and keep organized.

Track improvement over time

Keep the goals of resurrected user retention in mind as you form your metrics:

• Trigger dormant users to become resurrected users
• Get resurrected users to become current users

We suggest tracking these metrics over time to measure your progress

• The proportion of your active users that come from resurrected users (which you can do via Lifecycle).
• Long-term retention of resurrected users to see what percentage become current users. You want to avoid strategies that only result in short-term spikes in activity.
• Efficacy of any reengagement campaigns (push notifications or emails). Keep track of open and click through rates, as well as downstream metrics like retention and critical funnel conversion rate for each campaign.
• Stickiness of critical events.
• Conversion rate over time through your critical path funnel.

A resurrected user is someone who is active in the current period, was not active in the previous period, and was active at some point before that.

By analyzing your resurrected user retention, you will learn how you can:

1. "Resurrect" or reactivate dormant users
2. Get them to become current users of your product

Resurrected user diagnostic checklist

Run through the metrics below to get a baseline understanding of your resurrected users. Refer back to Ch. 4 for a refresher on any of these methods.

• Create a cohort of your resurrected users
• Plot the baseline retention curve of resurrected users
• Segment the retention curve by user properties
• Measure conversion through your critical path funnel
• Identify common user flows
• Measure stickiness for your critical event
• Measure session metric

PRO TIP:
Compare the retention of resurrected users to that of your current and new users. This will show you how your resurrected users currently perform relative to these other two groups and how much effort you want to devote to resurrecting users

Checklist item 1: Determine the opportunity size of resurrected users

Answer these 2 questions to get a sense of whether resurrected users can be a good source for boosting overall retention for your product.

1. What percentage of your active users are resurrected? (You should have already calculated this breakdown in the Chapter 3 Worksheet: Your Retention Lifecycle.)
2. How many potential resurrected users do you have? Calculate the number of people who used your product sometime in the preceding 6 months, but have not used it in the current time interval.

Checklist item 2: Resurrected user behavioral personas

Identify any behavioral personas within your resurrected users and list them here.

Checklist item 3: Identify triggers of resurrection

Remember, you could have internal or external triggers of resurrection (Section 7.3). Here are a few ways to identify triggers:

• Look at session utm parameters, referrer, and attribution data to look for common sources, like an email campaign or ad.
• Look for events within your resurrected behavioral personas that could have triggered resurrection.
• Analyze the paths of resurrected users, using Amplitude’s Pathfinder or a similar path visualization, to look for patterns in what users are doing when they return. List any triggers you find here.

Checklist item 4: Take action: hypotheses & next steps

Ask yourself these questions as you form hypotheses and come up with experiment ideas.

• What percentage of your active users currently come from resurrected users, and what’s the potential active user growth you could get from increasing the number of resurrected users?
• Are there any ways you can tailor the experience for resurrected users to increase the chance that they re-engage and become a current user?
• Is there an opportunity to trigger more users to resurrect that will provide an overall lift in your retention and other core metrics?
• What are effective triggers that you can experiment with for resurrecting dormant users?

Checklist item 5: Metrics for tracking improvement over time

As you start testing some of your hypotheses and trying out ways to improve your resurrected user retention, it’s important to keep track of your metrics to see what is and isn’t working.

Keep the goals of resurrected user retention in mind as you form your metrics:

• Trigger dormant users to become resurrected users
• Get resurrected users to become current users

We suggest tracking these metrics over time to measure your progress:

• The proportion of your active users that come from resurrected users.
• Long-term retention of resurrected users to see what percentage become current users. You want to avoid strategies that only result in short-term spikes in activity.
• Efficacy of any re-engagement campaigns (push notifications or emails). Keep track of open and click through rates, as well as downstream metrics like retention and critical funnel conversion rate for each campaign.
• Stickiness of critical events.
• Conversion rate over time through your critical path funnel.

The Amplitude Guide to Customer Retention: 40+ Resources to Increase Retention

The Amplitude Team

Customer Retention 101

The Amplitude Team

The Billion Dollar Mind Trick: An Intro to Triggers

Jason Hreha & Nir Eyal, author of Hooked

The Psychology of Notifications: How to Send Triggers that Work

Ximena Vengoechea & Nir Eyal

How to Bring Inactive Users Back From the Dead

Ty Magnin, Appcues Blog

Empty States (emptystat.es)

Website that collects empty states from all kinds of products – great for inspiration!