Return on (Product) Intelligence
Amplitude product evangelist John Cutler digs into the challenges of forecasting product intelligence ROI, and why it is important to not lose the forest through the trees.
Like many B2B SaaS companies, at Amplitude we frequently get asked to provide prospect-specific ROI estimates for our product. “What will our return on investment be if we purchase your product?” I have extremely passionate teammates working on this challenge, and it is challenging. Why? It’s easy to point to existing customers and talk about outcomes, and we have the luxury of having lots of amazing success stories to choose from (see Patreon, Calm, Instacart, Blue Apron). But it is a lot harder to estimate ROI for a company that isn’t currently using the product (even when you’re confident it could be a game-changer for them).
There’s an extra layer to this I find very interesting. The core problem that Amplitude solves for is the very problem that leaves prospects asking for an ROI estimate. It’s a bit of a Catch-22. Let me explain. A company that has a reasonable sense of their decision quality, a sense of the factors contributing to customer LTV (lifetime value), and has specific data-informed wins (and losses) to point to for reference, has a much easier time deciding for themselves what the ROI might be for increased speed-to-insights, depth/breadth of insights, and improved collaboration.
Some companies that ask us for ROI estimates are missing key data points (e.g. CAC, LTV, new user activation rate, current monthly retention). Other companies track these key product metrics, and watch them move, but are failing to close the learn-build-measure loop. They can’t link their product decisions to impact. These are the types of companies that get the highest ROI from investing in us, but are the most difficult to quantify.
Put simply, to have some idea about what improved decision quality and velocity might look like in your environment, you’ll need to have some sense of your baseline. And if you don’t, then that’s your first order of business. Here’s a good initial exercise. Pick a metric related to retention, engagement, expansion, time-to-value—whatever—and reflect on your team’s ability to impact that metric over the last 6-12 months. How did you do? Was that easy? Hard?
To have some idea about what improved decision quality and velocity might look like in your environment, you’ll need to have some sense of your baseline.
A variation on this thought exercise (especially if you’re having difficulty with the question above) is to ask how much you’d be willing to pay for a better sense of your own ROI as a team! It’s a decent framing for the value of product intelligence. But it is also an important prerequisite to the Amplitude (or similar product) ROI discussion.
Another angle here is about having first-hand experience in what is possible and where your intuition can go wrong. I’ll give you an example. I worked on a team once that spent almost 14 months working on optimizing a flow. We had ups and downs. After some missteps in the beginning—namely misreading our target customer segment—we hit our stride and made some big initial step-change improvements driven by key insights. Then things got way harder and humbling (as they tend to do after repeat efforts and early wins). It got to the point where most of our experiments failed. We knew it was time to move on.
These eye-opening experiences make me wince when I see a team of one-hundred engineers and designers hire five more engineers for $1,000,000/year, but be unable to answer basic questions about their decisions, their users, and their product. Sure, talented engineers and designers are indispensable. But you’re wasting a good deal of that potential without a solid feedback mechanism. People leave when they can’t see the impact of their work. Products like Amplitude are amplifiers and force-multipliers. With them you:
Products like Amplitude are amplifiers and force-multipliers.
- Evolve more coherent value creation models
- Focus on more promising points of leverage
- Better understand the impact of your interventions/experiments.
And, you get better and better at it.
I met a small team recently of 6-8 people, that I am fairly confident was running circles around another team—in a similar domain—of 60. Was their advantage 3x or 5x? I don’t really know the exact multiple, but it was in that vicinity. Many Xs. The smaller team was executing on almost as many decisions/interventions each quarter (despite being 1/10th the size) and had a demonstrably better track-record of generating outcomes. They were a learning machine, and they seemed to be getting stronger. Unless something changes, the smaller team’s company will “win”. With product development, the spread between the laggards/incumbents and the trailblazers/disruptors is extremely wide. It’s that kind of improvement we’re focused on here at Amplitude.
With product development, the spread between the laggards/incumbents and the trailblazers/disruptors is extremely wide.
Wrapping up…
If you are hoping to bring Amplitude into your organization and need an ROI estimate, we’ll happily dig into your product and business and do our best to help. We support our champions, appreciate partnering on the exercise, and enjoy the challenge. If you end up buying, we’ll work tirelessly to help make you successful. But try not to lose the forest through the trees. Often these exercises are bounded by your current reality…the data (and access to data) you have now. The goal isn’t a “modest lift”. The goal (and our goal as a company) is a big leap. We look forward to working with you on that!
John Cutler
Former Product Evangelist, Amplitude
John Cutler is a former product evangelist and coach at Amplitude.
More from John
