In this chapter, you will learn:
Over time, the North Star should result in improved business performance and happier customers. Even before that, though, you may observe changes within your company. The following are some signals that the North Star framework is working for your organization and your product:
At some point, a business may recognize that the North Star is no longer effective. Maybe your market has changed or your business is at a different stage. No matter the reason, at some point you could find that your original North Star no longer predicts business success.
Recognizing this and making this change requires insight, transparency, humility, and communication. At Amplitude, we found ourselves in this situation after implementing our original North Star, Weekly Querying Users.
Changing Amplitude’s Original North Star
*This section was written by Amplitude’s VP of Product Justin Bauer.*
In early 2017, we were a small product development team of about 20 people. Though we were still in our early stages, we knew it was important to have a metric that we could use to drive clarity, communicate progress to the rest of the organization, and hold product accountable to driving business outcomes for the company.
As a product analytics solution, our main value proposition was helping product managers and analysts answer questions, so we decided to use a ‘query’ as our indicator of value. That was important to us because our main differentiator from other analytics products was that Amplitude users didn’t just view dashboards, they explored the data deeply to understand what drives user behavior. So, we aligned around a metric we called weekly querying users (WQUs) as our North Star. This worked really well for us—we found that not only did WQUs reflect customers getting value out of Amplitude, but that this was also a leading indicator of our ability to retain and expand accounts to grow our business.
Over time, however, we recognized that this metric wasn’t truly reflecting the type of impact we wanted to have with our customers. Our mission isn’t to help users build better analyses—it’s to help companies build better products. And building a product is a team sport. By focusing only on users who query, we realized we were limiting the potential of our solution.
Changing a North Star Metric, especially at a company as analytically oriented as ours, is no small task. We started out doing what we do at our North Star workshops: examining our product strategy.
One of the changes we had made was deepening our focus on collaboration and exploration to help customers get to impact faster. By customer impact, we meant completing the “Build > Measure > Learn” loop (we’re big fans of loops at Amplitude).
We dug into our data to figure out how we could best represent a user completing the loop. We looked broadly across a multitude of factors: everything from Amplitude engagement and feature adoption to customer product velocity and usage of instrumentation.
We also did a deep network graph analysis to understand the relationships between users in our product. By looking at network structures and how they evolved over time, we could see how connectedness differed for our most successful customers compared to those who hadn’t fully adopted the platform.
Based on all of this, we ran correlations against our contract data and compared that to our existing WQU metric. Ultimately, we landed on a definition that we felt was simple enough for people to understand, yet robust enough to be a strong driver of revenue for Amplitude and capture value delivered. We call it the “Weekly Learning User” (WLU).
A WLU is a user who is active and shares a learning that is consumed by at least two other people in the previous seven days. It represents our most valuable user persona—the Advocate—who shares context to drive decisions and take action in an organization.
In addition to adapting our North Star, we also wanted to identify the key leading indicators of this new metric so that we could align our teams to that strategy. We identified three new Input metrics for our team to set goals against: Activated Organizations, Broadcasted Learnings, and Consumption of Learnings.
Activated Organizations: Activated Organizations (AOs) are organizations that have reached at least five WLUs. We have found that once we reach this threshold, there is a high likelihood that this customer will continue to grow with us.
Broadcasted Learnings: A chart, dashboard, or Notebook is considered to be a “Broadcasted Learning” (BLs) when it is consumed by two or more people within a seven-day period. By measuring the total number of high-quality learnings, we are able to get great insight into the health of our accounts.
Consumption of Learnings: Consumption of Learnings (CoLs) measures the total reach of all BL. This is our best proxy of how deeply an insight is shared across the organization. CoL measures the total reach of BL in an organization within a seven-day period.
We believe our new North Star Metric better reflects our current product strategy—the value that we are delivering to our customers—and is a good leading indicator of revenue for the company. In the past six months, we’ve seen a 50 percent increase in the number of WLUs per active organization.
We’re now exploring how to integrate our North Star into other parts of our company. For example, how can our Customer Success team use WLUs, BLs, and CoLs to track the health of a partnership? Can we trigger training sessions or campaigns with customers based on their likelihood to activate? How can we use these metrics to help our Account Management team identify customers that are likely to expand so they can establish a relationship with that account months before renewal? These are questions we’re looking forward to tackling as we build upon our new North Star Metric.
We believe our new North Star Metric better reflects our current product strategy—the value that we are delivering to our customers—and is a good leading indicator of revenue for the company. In the past six months, we’ve seen a 50 percent increase in the number of WLUs per active organization.
We’re now exploring how to integrate our North Star into other parts of our company. For example, how can our Customer Success team use WLUs, BLs, and CoLs to track the health of a partnership? Can we trigger training sessions or campaigns with customers based on their likelihood to activate? How can we use these metrics to help our Account Management team identify customers that are likely to expand so they can establish a relationship with that account months before renewal? These are questions we’re looking forward to tackling as we build upon our new North Star Metric.
In our experience working with product teams, many organizations trying to implement the North Star Framework succumb to what product development expert Jabe Bloom describes as *gap thinking*, which causes them to lose momentum and revert to old patterns.
In a gap thinking model, teams examine the current condition, envision a future state, and attempt to define and close the gap between the two.
The major problem with gap thinking, as it relates to any change effort, is that teams become vested in a fixed endpoint that never materializes. They inadvertently devalue the present. There’s a lot of long-term planning and execution and long feedback cycles. When they don’t achieve the future—when they don’t close the gap—they lose steam and give up.
If you find that your efforts to identify and implement a North Star for your product are losing momentum, you might be guilty of too much gap thinking.
Bloom contrasts gap thinking with present thinking. In present thinking, you recognize the reality of the present and work to change it. Present thinking asks, “Where are we? What do I need now? How do I improve the current way of working?”
Teams that successfully implement frameworks like the North Star Framework are never “done.” They are always learning and grappling with uncertainty. The real muscle to develop is continuously checking to see whether your North Star Metric and Inputs represent your current beliefs, product vision, and product strategy—then refine accordingly.
To make the shift to present thinking, consider Jabe Bloom’s present *thinking* questions:
Applied to the North Star Framework, the present thinking questions look like this: