Breaking the Vanity Metric Cycle

After tracking over 1 trillion actions, we showed that only 20% of users returned to an app 31-60 days after first use. Although this is better than the oft-cited “80% of users churn in three days” statistic, mobile churn is still huge, especially after first use. That’s a steep decline of users within the first few days, no matter how you cut it. And that’s why it’s often so tempting to shy away from stats like mobile churn rates in favor of gentler, nicer metrics like downloads. Vanity metrics usually that tell us that we’re succeeding, even if they probably mean nothing. We’ve written a lot about how to understand your retention and ditch vanity metrics. But breaking free of worthless metrics is hard because it is breaking a psychological reward, not just adopting some new stats. If you actually want to drive your company towards success, you need to be able to take an honest look at what is going on.

Prime yourself for success

A key factor in breaking the vanity metric cycle is understanding that building your business means honest self-reflection. For the short term, make finding the truth the central focus of your team culture. “The best way to execute and manage a long term goal starts with setting up the right expectations from the beginning,” says Ask for Task founder Muneeb Mushtaq. That means establishing a precedent that if something isn’t useful — like vanity metrics — it’s not valued. Everyone wants to feel like they’re working on a product that’s doing well, but focusing on the process makes that wellness possible. Ensure that your entire team takes ownership of the gritty parts of becoming successful. For the long term, flesh out exactly what your larger goals are. If you know where you want to go, you will realize that padding your stats isn’t going to get you there. Instead, look for how you can take actionable steps based on your metrics to pave the road ahead. Don’t be surprised when you drop vanity metrics and all of a sudden your data doesn’t look so good, and keep your long term goals in your crosshairs.

Be a special snowflake

Vanity metrics look good for everyone, and you aren’t everyone. Your business doesn’t do the same things as every other business, and you need metrics that reflect the actuality of your products. While we can’t tell you exactly which stats are best for your business, we can tell you that they are:

• Actionable. A metric is actionable when it gives you information that allows you to make a decision. Usually, this means that you will be able to predict a behavior using the metric. If a metric can’t help you see what is going on with your users, it isn’t worth a penny.
• In cohorts. Cohort analysis is about identifying groups of users, and seeing how those groups behave. Monitoring different cohorts will help you better understand your users, which will help you retain them — the ultimate goal of any mobile app.
• Tailored to your product. Understand what makes a successful metric for your product. For example, your app might have a completely different baseline for user engagement than someone else’s. If you’re a messaging app, you want people opening the app multiple times a day. But if your app records workouts, two or three days between opens might not be a big deal.
  Once you’ve identified the most important metrics for your business, use them to ruthlessly set and pursue internal benchmarks of success. Working towards goals you know will benefit you is a much better pursuit than chasing things like pageviews — not to mention a much better use of your time.

Empathize with your users

The uncomfortable truth is that, once your product is in the world, it isn’t yours anymore. It belongs to every single person who uses it, and they will use it how they want. However, you need to stop thinking of your users and their actions as a black box. [Tweet “Once your #product is in the world, it isn’t yours anymore. “] Vanity metrics make users feel like a mystery. Throw them out to empathize with and think like your users. Trying to best model their behavior is what your analytics should strive to do. An easily digestible example is Tesco’s quest to get customers to shop more. The UK grocer did a large-scale purchasing analysis and started sending coupons — but not the coupons they expected. For example, rewards shoppers who bought diapers were sent coupons for baby wipes. But they were also sent coupons for beer, because “[d]ata analysis revealed that new fathers tend to buy more beer [than before…].” Their coupon redemption spiked to >6% above the industry standard 1-2%. By the way, that was a cohort analysis, one of the best ways to dive into your user behaviors. Tesco was able to isolate male users who were buying baby products and use that cohort to find a predictive behavior. Once you know what customers are doing, you can alleviate their pain points, or tweak to make their good experiences even better. Or, like Tesco, you could give customers things they likely don’t even realize they want yet. Embracing the right, actionable metrics will allow you to do this. However, if you ignore digging into user analytics for charting things like “downloads,” then you aren’t empathizing with your users — you’re treating them like numbers, not people. If you aren’t treating users as people, one day you’ll turn around and they’ll be gone.

Help yourself

At the end of the day, why are you building an app? Probably because it offers someone a service that helps people live a better life, even in a small way. Maybe you want to create better music streaming, an easier commute with an addictive game, a quicker way to connect with friends, or safer traveling. If you can’t take a hard look at your analytics to achieve those goals, your app will become just a nice idea you once had. You need to foster a mentality that allows you to tackle the hard truths to push your company in the right direction. The success you will build with actionable metrics will feel better than any vanity metric numbers ever could. [Tweet “Build your success with actionable #metrics, not vanity metrics.”]