Find your company's inflection metrics with Compass

Compass helps you identify behaviors that predict retention or conversion. Compass identifies inflection metrics, which capture the moments when a user reaches a critical threshold in your product. These metrics help drive user growth.

For example, Facebook found early on that adding seven friends in the first ten days was the strongest signal of long-term retention. More recently, Netflix published an analysis of how many episodes per TV show it takes for a viewer to get hooked.

Before you proceed, Amplitude suggests you read the Help Center documentation on Compass first. The rest of this article assumes a general understanding of how the analysis works.

For ease of reading, Amplitude uses the new user/retention use case as the example. You can replace new user with any base cohort, and replace retained user with any target cohort.

To find an inflection metric, first decide your target cohort. The inflection metric often centers on encouraging new users to become retained users. In the examples in this article, the base cohort is new users and the target cohort contains retained users.

A base cohort is the initial set of users you're analyzing (for example, new users or logged-in users). A target cohort is a set of users that have completed a targeted action (for example, retention or conversion).

This is a common use case, and it's the default setup for Compass charts. You can edit your chart so that it reflects your specific analytic needs.

When looking for an inflection metric, remember that it isn't absolute. It doesn't mean a user converts at that exact point. Instead, it suggests the type of behavior you want your organization (for example, the product and marketing team) to encourage in your users.

Get started with Compass

The best way to begin using Compass is to ask yourself which events might be good predictors of retention. After you select an event to analyze, start thinking about correlations that might reveal something interesting about user behavior.

Proportion above threshold

The proportion above threshold tells you how many new users triggered the event in their first N days. This matters because the sample of users meeting the threshold must be large enough for Compass to understand how well the event correlates with retention.

One way to change the proportion is to increase the number of performance days in the window (Amplitude allows between one and seven days). More performance days give users more time to reach the threshold, which increases the proportion. If you're investigating an event property, consider looking at the complete event, which may have a higher proportion above the threshold.

There's no perfect proportion above the threshold. Too low and you can't get many new users to perform that event; too high and you don't have any room for improvement.

In some extreme cases, a low proportion above the threshold can still produce a high correlation. For example, a web application that has high traffic but forces login for all new users.

This metric matters because it accounts for the balance you need to find your inflection metric. Returning to the Facebook example, getting a new user to add one friend isn't a great choice for an inflection metric, because most users do that already. But getting a new user to add 100 friends, while highly correlated with retention, is hard because few users actually reach that level.

True positive ratios: PPV and sensitivity

If you have a reasonable proportion above the threshold, view the correlation between reaching the event frequency and retention. Look at the positive predictive value (PPV) and sensitivity.

PPV looks at the ratio of users who reached the event frequency and retained (true positive) to all users who reached the event frequency (true positive + false positive). Sensitivity looks at the ratio of users that retained and reached the event frequency (true positive) to all users retained (true positive + false negative). You want both to be high.

For more information about true positives, false positives, and other values in a contingency matrix, refer to Confusion matrix on Wikipedia.

Example 1: High PPV, low sensitivity

Imagine PPV is high but sensitivity is low. The event predicts retention, but few new users reach the threshold. The event is a promising candidate for experimentation to see if you can encourage more users to trigger it. The result also means another inflection metric might exist that you haven't looked at yet, since people who aren't reaching this frequency are still retained.

Example 2: Low PPV, high sensitivity

The event frequency captures many of the retained users, but the total retention for the product is likely low. This isn't a good candidate for an inflection metric, because either the product's retention is low or a high percentage of users meeting the event frequency aren't retained.

True negative ratios: NPV and specificity

The inflection moment should be a positive predictor. You also want to ensure that when a user fails to reach the threshold, the failure is a negative predictor of retention, in other words, churn. Amplitude captures this through the negative predictive value (NPV) and specificity.

NPV looks at the ratio of users who didn't reach the event frequency and didn't retain (true negative) to all users who didn't reach the event frequency (true negative + false negative).

Specificity looks at the ratio of users who didn't reach the event frequency and didn't retain (true negative) to all users who didn't retain (true negative + false positive). As in the examples above, you want to maximize both of these values.

Example 3: High NPV, low specificity

In this example, one of two things is likely happening. Either the PPV is also low (as in Example 2), or the proportion above the threshold is so high that there's no room for improvement by encouraging this action. Neither makes a great inflection metric.

Example 4: Low NPV, high specificity

Here, either the sensitivity is also low (as in Example 1), or the retention is so high that few users remain to convert, which is a good problem to have.

The Compass analysis tries to uncover event frequencies that maximize the upper-left (true positive) and bottom-right (true negative) quadrants of the contingency matrix. If you're familiar with statistics, this minimizes the Type I and Type II errors.

These inflection metrics balance all five of the detailed statistics in the contingency matrix. Depending on the type of product, a good correlation falls in the range of 0.2 to 0.4, depending on the number of performance days (1 to 7) for the event.

Check that the sample size is large enough to draw conclusions. There's no magic number, since it depends on your total user volume, but you can see the effect of sample size by clicking the blue +- number (the 95% confidence interval) next to the correlation. Change the date range to increase the sample size. You can use up to 90 days of data.

Compass exposes correlations from your data, which are hypotheses you can test by making changes to your product or lifecycle marketing. The only way to prove a causal relationship is to run an A/B or split test to isolate those changes. Read more about how you can analyze A/B test results on Amplitude.