Cohort Retention Analysis: Reduce Churn Using Customer Data

Originally published on July 28, 2022

Cohort analysis answers a business question about how a specific group or segment of users has interacted with or is expected to interact with a product. Breaking users' behavioral data down into cohorts helps teams better understand what users want from their product.

Cohorts are groups of users that share specific characteristics and usage patterns over a period of time. These could include things like usage time, features added to an account, or the number of goals completed. Cohorts help you segment your user base and collect data about how different groups interact with your product throughout their lifecycle.

One useful application of this technique is cohort retention analysis: a process that seeks to isolate indicators of retention so a product team can guide users through successful product interactions. The data you obtain will help you understand what causes new customers to stick around and some of the common reasons why they churn.

What is cohort analysis?

Cohort analysis is a method of data-driven reporting that seeks to understand user behavior by grouping audiences around shared traits. User cohorts may be based on demographic characteristics, behaviors, or both. For example:

• Users who own iOS devices
• Users who logged in every day last week
• Users who own iOS devices who logged in every day last week

Cohort analysis is typically prompted by business questions regarding engagement, conversion, or retention.

• Engagement is the term for actions people take within your app. That could be leveling up a character in a game, sharing a workout with the community in a fitness app, or playing a song in a music app.
• Conversion is the ultimate goal. Cohort analysis can be used to judge whether different incentives for conversion, like new features or discounted rates, are effective.
• Retention is the opposite of churn, and it measures how many users return to your product over a specific time period.

Try creating a cohort comparison in Amplitude’s self-service demo or explore cohorts within your own data using our free starter plan.

In the example chart above, you can see the user journey for two different cohorts—users who shared a song (green) and those who did not (blue). The cohort of users who shared a song had higher engagement and conversion.

Now, let’s take a closer look at how cohort analysis can be used to help product managers reduce customer churn and improve retention rates.

Using cohort analysis to improve customer retention

Cohort retention analysis can help you dig deeper into your churn metrics, which are a key indicator of a company’s outlook. If you want to increase your retention, simply calculating customer churn rates isn’t enough (though we do have a churn calculator to help with that). Data regarding how long users are sticking around and what they’re using your product for is essential to optimizing the customer experience.

For example, suppose you are a product manager of a music app like Spotify, and one of your primary goals is to increase user retention. Your company may have hypotheses about what makes certain users stay with your app—and those theories likely inform your marketing efforts, product roadmap, and onboarding flow. However, a hypothesis is no match for actual customer data.

Running a cohort analysis helps you test your presuppositions against reality. You can see how long certain user groups are sticking around and use behavioral data to find similarities between loyal customers and those who fall off in the first week. You can even run mass analyses to find the customers who are most likely to take specific actions linked to retention.

You won’t get all of these answers from one analysis but from a series of cohort retention analyses. Let’s dig into the types of cohorts PMs use and what you can learn from each.

3 types of cohort data and how to use them

You can collect three types of cohort data for your retention analysis: acquisition, behavioral, and predictive. Each type of cohort analysis works for specific use cases.

1. Acquisition cohorts: Track customer retention over time

Acquisition cohorts give you information about your customers’ lifecycle—specifically, how long it takes them to churn after their acquisition date. This information can help you identify churn patterns or marketing campaigns that result in higher conversion and retention rates.

Savvy product managers turn to acquisition cohorts to evaluate how well their onboarding processes show their product’s value. The sooner new users have that aha moment, the more likely they are to be retained.

Say you have a music app that is experiencing a churn problem with users dropping off each day:

The users in the retention chart above are split into daily cohorts—users who signed up on the same day. You can see that 11,528 users signed up for your music app on July 16, and Day 5 retention was 49.7%. So one in two users who signed up on July 16 were still active users in the app on the fifth day after first using the app.

The best way to visualize this information is to turn it into a retention analysis curve, which shows your retention for these cohorts over time. When you chart your data like this, it becomes easy to see when users are leaving your product.

Learn about the acquisition cohorts in your data by using Amplitude’s free starter plan.

This retention curve immediately tells you something important: About a third of users stop using the app after the first day. After that initial drop, the curve decreases steadily, leaving only a little more than 25% of the original users still active in the app on day 30.

That’s not great (although it is common—Amplitude data shows that the average app loses more than 80% of users within the first month). Early retention is a significant issue. A curve like this indicates that users aren’t getting the core value out of the app quickly enough, so they are leaving. Now you know it’s time to improve your onboarding process so new users have a reason to stick around.

The limits of acquisition cohorts: Information without explanation

If your app has the above retention curve, you immediately want to figure out what you can do to boost your retention. However, acquisition cohorts alone don’t give any information about how you can improve the user experience to retain your users. You can’t isolate specific behaviors or user properties.

Acquisition cohorts are great for showing trends and telling you when people are churning, but they don't tell you why. That’s where behavioral cohorts come in.

2. Behavioral cohorts: Discover which behaviors drive retention

Behavioral cohorts are a custom segment of your audience based on any combination of past behaviors or user profile properties. This type of cohort analysis looks at actions within a certain time frame to identify users who interact with your product similarly.

The combination of behaviors and profile properties collectively make up a behavioral cohort. That analysis usually reveals how users engage with your product and how that user engagement affects things like retention, conversion rate, or other key indicators that matter to your business.

When reworking your user onboarding to boost retention, you’ll have to identify the most effective way to do so. Rather than choosing what to work on based on anecdotes or random choices, behavioral cohorts enable you to decide on an approach systematically and quantitatively.

Finding, inverting, and combining cohorts

You can create different user cohorts for your music app from actions like playing a song, searching for an artist, or joining a community.

Say you wanted to see the retention of users who favorited songs in the app. You can use behavioral cohorts to look at retention for new users who favorited three or more songs:

Discover behavioral cohorts within your own data by getting started with Amplitude for free.

While almost 40% of all users (blue) churn within one day of using the app, only about 15% of users who favorite three or more songs (green) drop off after day one.

Now that you know how retention changes for users who interact with their favorite feature, you can also look at how it changes for those who do not. Below is the inverted cohort: users who did not favorite a song:

This cohort has worse retention than most—more than 75% of these users churn after the first day.

From this simple visualization, it’s clear that getting people to favorite songs early in their experience helps them discover the app’s core value, meaning they are more likely to continue as users.

However, favoriting songs may not be the only behavior that predicts retention. You can combine cohorts to see whether certain actions have an additive effect.

For instance, your music app also has a feature that lets people join communities based on their favorite genres.

The initial retention for users who join communities (purple) is similar to users who favorite songs (green), but it’s slightly better by the end of Day 30 and much better than all users (blue).

What about users who favorite songs and join communities? Using Amplitude, you can filter your actions to combine these two cohorts:

Users exhibiting both behaviors are far more likely to keep using the app in those first few weeks. At the end of the first week, retention is above 75% for the favorite + community cohort (blue), while it’s under 25% for users without either of these behaviors (green).

The limits of behavioral cohorts: Correlation, not causation

The data you’ve gathered from your cohort analysis suggests your onboarding should teach users how to join communities and add favorite songs. Users who follow either of the prompts will be more likely to stick around than those who don’t, and users who follow both are overwhelmingly likely to stick with your app. Right?

Not quite. Just because people who favorite songs and join communities churn less doesn’t mean that driving people toward these behaviors will automatically reduce your churn rate.

That’s because correlation does not imply causation. Favoriting songs and joining communities could be a sign of higher user engagement, not the cause of it. To determine causation, you might A/B test different onboarding flows that emphasize favoriting songs to see if it increases retention.

3. Predictive cohorts: Increase your marketing ROI

Predictive cohorts use behavioral data to predict what users will likely do in the future. This type of data is best for determining which users to target with a marketing campaign or implementing churn reduction campaigns that target less engaged users.

Let’s return to our music app example and see how predictive cohorts can determine how likely a group of users is to purchase a song in the future based on their behavioral cohort.

You might start with a known engaged group—for instance, new users who shared a song in the last 14 days—and run a prediction analysis. After about 30 to 60 minutes, the model will create and rank specific cohorts based on who is most likely to take the action you’re exploring (e.g., purchasing a song).

Define your Starting Cohort and Future Outline—like purchasing a song in Amplitude.

You now have data in a simple cohort chart to inform an effective business decision. For example, you could target a marketing campaign to the top 5% of users who are most likely to buy a song. This could be a push notification, an email, or an SMS alert to encourage them to make their first purchase.

At the same time, you could look at the 20% of users the predictive analysis determined were least likely to make a purchase and choose not to assign any marketing dollars to this low-potential cohort. Alternatively, the group of users simply needs a different or greater incentive. Maybe sending a 50% discount coupon their way will be an offer so compelling they can’t refuse it.

The limits of predictive cohorts: Mass data, opaque conclusions

Because predictive cohorts use machine learning to determine how users are likely to interact, you need a large base of active users (or a large pool of behavioral data collected from prior users). This type of analysis works best if you have a cohort size of more than 100,000 users. If your cohort has to be 100,000 strong, your user base has to be larger than that! After all, many of your music app’s new users won’t have shared a song within the last 14 days.

The other downside of predictive cohorts is the lack of information regarding why users are expected to behave in a certain way. Machine learning models are so effective because they can analyze hundreds of data points at the same time. However, the cohort analysis won’t tell you which of these hundred data points affects whether a user will purchase a song from you or not. While you can take action based on the results of the predictive analysis, you can’t take action that would move more people into that “likely song buyers” group.

Because each type of cohort analysis has its strengths and weaknesses, all three will stay useful as your product grows and evolves. Finding success with cohort analyses means being able to determine which method will give you the most actionable results.

How to build a retention strategy using cohort analysis

The most powerful aspect of cohort analysis is that you’ll not only see customers leave and when, but you can also start to understand why your customers leave your app—so that you can fix it. Here’s how to launch a cohort retention analysis and take action based on the results.

Step 0: Explore existing retention and cohort data

If you already have data regarding customer churn rates and/or cohorts, you can start your analysis from a more informed place. (If you don’t, feel free to skip to step 1.) Now is the time to look at what you do know and determine what you don’t know.

For example, you may have had a huge drop in active users or subscribers last month. Clearly, something happened, and it’s up to you to determine what. Is this related to acquisition (you had a large wave of sign-ups eight months back, and users usually stick around for seven months) or behavior (you rolled out a new feature and sunsetted two others, and users disliked one of the changes)?

On the other hand, you may just have a vague goal to “increase retention rates” because your customer acquisition cost is creeping up, and revenue isn’t keeping pace. In this situation, it’s best to start from a place of exploration. Do you see any big drop-offs in your user retention curve? What about if you split the data into acquisition cohorts?

Maybe you’ve looked at your data and can’t find any clear indicators either way. In that case, move on to step 1; you haven’t lost anything. You’ll just be building a bit more than some others have to.

Step 1: Create a hypothesis

Every successful experiment starts with a theory of why things work the way they do. In a cohort retention analysis, your hypothesis will be about why your users are churning.

Those starting with an existing set of cohort data will be able to make a more informed guess, but even without the numbers, you know your product and user base well enough to come up with a plausible cause for the churn you’re seeing.

Your hypothesis will tell you what type of cohort analysis you’ll be running. If your hypothesis is related to when users start dropping off from your platform, acquisition cohorts are the tool you’ll use. You'll set up behavioral cohorts if it’s related to certain behaviors or interactions you think are correlated to retention. And if you’re predicting the effect a certain behavior has on later interactions with your app or product, a predictive cohort will be your best bet.

Step 2: Build your cohorts and collate data

For your first cohort retention analysis, you’ll likely want to start with an acquisition or behavioral cohort. Save the predictive cohorts for when you better understand your users and how they interact with your product, especially if you’re new to collecting and thinking about cohort data.

Since acquisition cohorts are defined by their join date, this type of retention analysis requires data points like sign-up date or account age and frequency of product use. If your company uses other measures of retention (like subscription status), gather that information as well.

Behavioral cohorts are a bit more difficult to build because you’ll have to decide which behaviors to look at. Most companies start by tracking what they consider introductory behaviors (uploading a profile picture, changing your username, choosing light or dark mode) and the aha moment (a task accomplished or benefit discovered). Look to existing resources like in-app onboarding flows or customer success managers’ new client checklists to pinpoint the most important behaviors.

Predictive cohorts rely on two things: a lot of data and one specific behavior. If you don’t have a large user base, you’ll want to stick with the other two cohort types for now. When running a predictive cohort analysis, you’ll want to define a behavior you think is key and then one you suspect follows it. For example, you may be looking at whether adding a profile picture (key) results in adding 10 or more friends within one week (following).

It’s easy to track these events using a tool like Amplitude (we’ll have more on that later!), and you’ll want to get started ASAP. The more data you have, the more users you can confidently assign to cohorts.

Step 3: Analyze your data

Once you’ve collected enough data to build cohorts of a significant sample size, it’s time to dig in. A well-crafted cohort retention analysis is easy to make sense of.

Acquisition cohorts are usually displayed in a table format, each forming a row. The columns list the percentage of users in a cohort who are still active at a certain period (typically measured in days or months) after signing up. Use whatever metric your company considers its key retention indicator to get those percentages.

What you’re looking for in an acquisition cohort is an anomalous drop-off in your retention rate. Any retention efforts that start on or after users reach this point will have diminished effectiveness. Additionally, it’s smart to compare your cohorts to see if there are any changes. Ideally, you’d see that older accounts had bigger drop-offs from period to period than newer ones, meaning your retention practices have been making a difference. If your younger cohorts are leaving at higher rates than those who came before them, that’s a problem that deserves your attention.

Behavioral cohorts have a similar layout if displayed in table form, but the columns list the number of people who take the action you’ve been tracking at a certain period after that initial interaction.

However, these cohorts are best understood through visualization as a line graph or bar graph—something Amplitude does for you. These graphs make it easy to compare your cohorts by color coding (color one for the group that took the relevant action, color two for the group that did not) and displaying the retention rate (Y-axis) over time (X-axis).

If you see a difference in retention rate between your two cohorts, it means you’ve found a behavior that at least correlates with users sticking around. If the two cohorts show roughly the same rate of churn, the behavior you were tracking does not correlate with retention.

Predictive cohorts are usually the easiest for users to analyze because they’re fully automated via machine learning. Once you’ve fed enough data to Amplitude, you can see whether your key and following behaviors are related.

Step 4: Interpret the results and form your next question

After getting a view of what the data says, it’s time to figure out what it means for your product. Then, you can make changes or run continued analyses to test different retention practices.

Information about when people are dropping off, like you’d gather from an acquisition cohort, can only tell you so much on its own. If people are dropping off early, your onboarding process may not be doing its job of showcasing your app’s value. If recent cohorts are churning more quickly than older ones, you may have changed your onboarding process in a way that hurt retention, or you may be bringing in users who aren’t a natural fit for your app.

Whatever it is, you won’t learn the reasons from another acquisition cohort. So, based on the trends you can see within and between cohorts, it’s time to start the cycle all over again at step 1. This time, you’ll be making a hypothesis that can be tested with a behavioral cohort.

Behavioral cohorts, on the other hand, tend to give more concrete information. The data tells you whether the behavior you were tracking correlates with retention. If so, you want your users to repeat it; if not, it’s time to hypothesize about another behavior that might be keeping people around.

Finally, predictive cohorts tell you where to focus your marketing and retention efforts. Seek out the user profiles that are most likely to complete your desired key action and nudge them toward it. You’ll use resources more efficiently when you’re not spending time on users who were always going to churn because your product isn’t what they want.

Step 5: Iterate your product and keep testing

Once you know which behaviors correlate with retention, it’s time to reduce churn by nudging your users to complete the surveys. You can also determine if there’s a causal relationship by using an A/B testing tool like Amplitude Experiment to look for changes in customer behavior.

As you continue to make these incremental changes, continue to compare your cohort retention rates. Look for your newer cohorts to stick around longer and have higher overall retention rates.

Amplitude: The best tool for cohort retention analysis

Modern markets move fast, and businesses that can’t make fast decisions based on accurate data stand to lose revenue. Without the right analytics tools, nontechnical teams needing data to make better business decisions find themselves relying on data analysts and data engineers. It’s a high-resource task that takes time to fulfill—time PMs don’t have to spare.

Amplitude empowers product managers and marketers to answer their own questions with its self-service cohort analysis capabilities. There are three ways you can use Amplitude to look at cohort retention data.

1. Build a cohort within any chart in Amplitude, like the Retention Analysis chart below. Here, you can select any combination of behaviors and profile properties, like users who favorite a song or join a community.

Building a cohort within an Amplitude chart is a simple way to perform a quick analysis.

2. Use a dedicated Cohorts section to build custom cohort definitions based on your particular parameters. This custom cohort can then be used in other charts. For example, the cohort below shows new, active users on iOS who shared a pop or rock song in the last 30 days.

Product managers and marketers can build more precise cohorts within the Cohorts section in Amplitude.

3. Build a cohort based on a single data point found in any chart. For example, you can identify new users from a July 26 product launch.

Amplitude makes it easy to create a cohort of users based on a specific data point within an existing chart.

Ultimately, a useful cohort analysis tool empowers non-technical teams to ask and answer questions. Putting that information directly in their hands gives them a better understanding of their product users and better data to drive business results.

Cohort analysis examples

You’ve seen what a hypothetical cohort retention analysis might look like. Here are some real-world examples of how Amplitude’s customers have used cohort analysis to produce business results.

Calm: Cohort retention analysis leads to 3x less churn

On a hunch, Calm used behavioral cohorts to test the retention of users who set daily reminders on its meditation app compared to those who didn’t use the feature. They discovered a 3x increase in retention for users who set daily reminders.

The reminder wasn’t easy to find, so there was a chance the users who liked the app most for some other reason simply dug into the menus further and found the feature. To test whether this was correlation or causation, Calm changed its onboarding tutorial to encourage some new users to set a reminder and left other first-time users as a control group.

That 3x retention rate held during the experiment, so Calm included the prompt to set daily reminders in its next app-wide update.

Cornerstone: Cohort analysis example for faster decisions

Cornerstone has transformed its product management workflow with Amplitude’s help. Before, product managers had to request data from engineers.

Those engineers would provide a cohort report with spreadsheets full of information, which required a full-time employee to pore over the information and glean insights that could lead to better business results.

The whole process could take days. Or weeks. Now, product managers can retrieve the same data in minutes and use the insights to make quick decisions.

Use Amplitude to launch your first cohort analysis

Now that you understand the ins and outs of cohort retention analysis and you’ve learned how to use them effectively, it’s time to try it out. Setting up your own cohort analysis with a cohort analysis tool like Amplitude is simple:

• Take a look at your retention with an acquisition cohort. That will show you when users are dropping off.
• Define events for a few of your app’s core user actions, and then create behavioral cohorts. Analyze these cohorts by comparing them, inverting them, and combining them.
• Use what you’ve learned from your acquisition and behavioral cohorts to generate a hypothesis about actions you can emphasize during a specific part of the customer journey to drive retention.
• Make changes to your app—using A/B testing with Amplitude Experiment if you have a high enough usage volume—to see whether driving certain actions in your app causes users to come back.
• Process your learnings and repeat.

With Amplitude’s cohort analysis tools, you can see the specifics of your customers’ behaviors and make data-driven decisions to enhance their experience with your product.