Where does product analytics fit in the realm of analytics tools on the market? What makes product analytics different from marketing analytics? How are product teams using analytics in their day-to-day work?
These are just some of the questions our Director of Product, Sandhya Hegde answered in a recent interview with Ryan Koonce, the CEO of Mammoth Growth, a growth agency that works with companies like DoorDash, Tile and Rinse.
In this 21 minute interview, Ryan spoke with Sandhya about her journey from venture capital to building products, how Amplitude is growing our own product and why product teams that use product analytics are winning. You can find a full transcript below.
(Hint: Skip to 3:45 to learn how Amplitude compares to BI tools and marketing analytics.)
Transcript:
Ryan Koonce: Hi everybody, I’m Ryan Koonce the CEO of Mammoth Growth, and I’m here today with Sandhya Hegde, the director of product at Amplitude. Thanks for coming in.
Sandhya: That’s right, thank you, thank you for having me here.
Ryan: So, you’ve had a pretty interesting background. How did you end up at Amplitude? Could you tell us a little bit about your history and what you’re doing here now?
Sandhya: Yeah, yeah, I started my career as an engineer building product and ended up in venture capital, and I worked with really great people at Sequoia Capital and Khosla Ventures and was kind of getting the urge to go back and build product again, and I was noticing a couple of things, right? I am definitely a student of the product game, and it was interesting to see how there was this major trend in every industry in the world right now where building great software applications was kind of like the new, you know, VEP. Right?
Sandhya: So, if you remember in like the late 90’s, 2000’s, everyone was saying “Oh, what’s my VEP strategy? I need to be on the web. What am I going to do?” And from there, you had this massive digital marketing industry that was born out of that trend. And you had this kind of CMO title that got created and marketing became a real competitive advantage for companies that knew how to do it well. And I see the same pattern right now with product where every company is asking themselves, “What is our strategy to have a great product experience for our customers?” The way customers feel about the product is your new brand. Product is how you distribute things in the world of premium, both across consumers and B to B. So it’s kind of become the only real competitive advantage left.
Sandhya: So, that means that this role of being a product builder was becoming more and more influential and interesting and Amplitude was kind of positioned right at the center of it, right? Because Amplitude as a company, we think we are really the only analytics platform that’s built ground up to help producting succeed. So, that’s kind of what attracted me to Amplitude. I was definitely seeing this trend as a VC and really wanted to, you know loved the team here, and really wanted to walk with them, to help them fuel this revolution.
Ryan: That’s great. What, can you talk a little bit about your specific role at Amplitude? What are you responsible for shipping for the product?
Sandhya: Yeah, yeah, so I’m the director of product at Amplitude, one of the things I’ve done is led work along with the team on a lot of the new products that we have launched. Kind of improvements that we are making to the cold platform over the last 12 months or so. And I’m also working on our product marketing strategy because what we realized was that the way product people buy is very different. There is no standard playbook for how you sell to them, how you market to them, and we are doing a lot of research to understand how product people make buying decisions and how we can help them be more successful. So it’s a really interesting opportunity to kind of wear multiple hats and think about that.
Ryan: Well in organizations, product people can be very engineering-specific or they can be very design-specific. And so you have to tailor that conversation to those different contingents.
Ryan: So, the analytics landscape has changed a lot over the last few years, and I guess Amplitude is about 4 years old. And, when you think back to maybe 2010, and you had Mixpanel who sort of came on the scene with the identify and track model, which was revolutionary at the time and moved us a long way from Google analytics, right? And we saw a couple players come into that space, and we have Amplitude now. You know, and I think on the other side of that there’s the BI, call them more BI tools versus analytics, like Looker, or Periscope or Mode. Could you talk a little bit about how Amplitude as a product analytics platform is different from something like on one side a Google analytics and on the other side, a visualization platform or however you want to define those other types of tools?
Sandhya: Yeah, yeah, I think that’s absolutely the right way to kind of cut the industry. The way we think about it is you have kind of what we call marketing analytics where you still want to have some of Google and Adobe to tie in the advertising audience like performance marketing, how that is working and using Google Ad Words and Google analytics in combination makes a lot of sense. So, that’s kind of like the marketing analytics piece.
Sandhya: And then there’s BI, where, which is all about historical reporting, right? Being able to report very accurately, exactly what happened over the last 12 months, 16 months, being able to visualize it anywhere you want, which is kind of the industry that Tableau has built, right? They are the leaders there and with the right amount of effort, you can set up the perfect visualization that helps explain to the executive team, the finance team, the sales team, what was the pattern on some data set over the last 12 months.
Sandhya: The role we play is very different and very new and unique, which is that the power that Amplitude offers you is for exploratory analytics. And it’s specifically customized to be used on product data. Understand exactly how users are interacting with your product, why they come back, why they don’t come back, what features have you built that have improved the user experience versus not, it’s really hard to do that right now. To be able to say “I shipped this x feature and because of that, my overall product experience has improved or my overall customer experience has improved”.
Sandhya: It’s actually a very hard question to answer and what we see is that often, you require data scientists to come in and do the work to answer some of those complicated product questions. And while that’s great, like it doesn’t scale. Right? So we see companies using data scientists to answer questions and sort of letting the data science team like build the machine learning algorithm the company needs, which is really their core job, right? No one becomes a data scientist to help a product manager answer his questions. That’s not their role, you become a data scientist because you want to come up with an amazing algorithm that will power the product. So, Amplitude just helps expand that data access for complicated questions so that you can understand what is happening in your product related to what features you need to double down on and keep pushing out product outcomes faster and faster. And today, the speed with which you make improvements to your product is do or die, right?
Ryan: Right
Sandhya: If your competitor is improving their product faster than you are you are basically giving them more and more chances to succeed, right? That’s kind of the difference between companies that are succeeding and failing right now.
Ryan: So, I’m gonna stop you there. When you talk about a Tableau for accessing a data warehouse or visualizing that data for a moment in time, I think it’d be really helpful to talk about what the difference is. Because we find that people call us and say “I need a data warehouse”.
Ryan: We say, well you’re not even counting one plus one equals two yet. I don’t think you need a data warehouse. You need to understand what is happening with your customers, and you need to understand what’s happening in your business and in your product”. I think it would be helpful from your perspective to hear the difference between that opportunity for visualizing data in a data warehouse and not even data science, it’s just “I need a data warehouse” versus “I can use Amplitude to answer those questions”. How do we talk about the difference to people?
Sandhya: Yeah, that’s a great question. I think when it comes to just being able to visualize your basic data, that’s kind of, that’s a pretty solid problem. I would say that there are so many tools on the market that will help you just get started. And you know, even Google Analytics like the free version will help you get started, right? Or you could just buy Redshift and slap Tableau on top, and you have a visualization solution. You can at least start reporting we had this many active users who, or this many web sessions that we can now count. And yeah, that’s basics, and of course Amplitude does that stuff too. The problem is that some of these other tools only do that, right? That’s where they stop. They do that very well, but they don’t let you answer, well yes, they’ll tell you this metric went from x to y, but they don’t tell you why, right? So immediately, if your next question is, “Well, I was expecting it to be way higher, why is it low?” you are completely blocked.
Ryan: Well, and the reason is, I think, because Amplitude allows you to access time series data whereas in a database, it usually is just the static moment in time.
Sandhya: Yeah, yeah.
Ryan: Is that a fair statement?
Sandhya: Yeah, you can do more with effort, but it’s a lot of effort, right? So what happens is, what we see is that with other tools, most people use it for historical reporting because with some effort, you can set up any time series chopped. But then, exploring on that, building on that, asking the five why’s, the five how’s, is impossible. So, you just stop there, you don’t do anything else.
Sandhya: But, with Amplitude, we have all this functionality around investigating the data. Saying “Okay, I have a retention problem. Tell me who are the customers who retained the least and who are the customers who retained the best? What are the differences between them? What are the features that the customers who retain the most associate with? What’s the most important thing in my product? Is it week one, is it just the week one experience that dictates everything that follows?” And that’s the kind of stuff that allows you to prioritize your work and focus on the right stuff early rather than late. But if you just want to kind of visualize the data you have to just get reporting started, I would just recommend free tools, like use a lot of the free tools that we live with in the market and when you make an investment, make sure you are getting depth and flexibility for the real monitoring investment that you are making.
Ryan: Well, and you have to invest in Amplitude in the sense that you have to set it up right in order for it to work.
Sandhya: Mm-hmm (affirmative), yeah.
Ryan: And so, there is a place where you decide that you’re going to be a data-oriented organization and you’re gonna make the effort to implement the events in the right way and put the right properties on different things and add traits to users so that you can pull those reports that are important.
Sandhya: Yep.
Ryan: You know, the other question I have is, how do you manage complexity? Because, as Amplitude becomes more powerful, it does become a little bit harder and harder for the average user to pull all the levers in the right way. What’s the plan to continually keeping it easy to use while also unleashing all the power that you can by having these features?
Sandhya: Yeah, so we use what we call an adaptive approach to product development and that’s kind of also what we always walk through with our customers about, like, how they can take an adaptive approach to product development. And, what that means is that when we release new features, we always start with skateboard versions, start with versions that only our power users can really touch and feel. No one else might even see it. If they see it, they don’t know how to use it. And that’s almost by design because that allows us to ship something that’s not perfect really fast.
Sandhya: And then we look at whether our hypotheses are true, which is are our power users interacting with it the way we expected them to? Are they getting the value out of it that we expected them to? And most importantly, are they coming back to it to use it again and again? And, as soon as that happens, we start figuring out how to simplify it, making it more intuitive, making it like a work flow that’s not just a general solution that you need to learn how to use, but it’s an intuitive part of the work flow. So, often what you see is this life cycle of development of a feature where it has gone from being extremely hard to use, but there’s a small community that loves it, adores it, relies on it. To it becoming a simple, universal part of the platform.
Sandhya: And, I think my favorite example of this is some of the really advanced formulas we have inside Amplitude, in some of our reports, you can set up really complicated ratios and rolling windows, you can do all of this stuff that’s definitely power user functionality. When we first launched it, you had to just write-type the formula, there was no other way to do it, but now we’ll have auto-completes, you will have drop-downs, you don’t have to know what it means, it will show it to you right there, and we kept investing in it because we could see that people who interacted with it got a great amount of value from it. And similarly, there are other features, which we have pulled back, or not invested in, or removed slowly from the platform because we spent a week learning that that was not the right approach to a problem that we still need to solve.
Sandhya: So for us, the act of instrumenting, setting up attributes, setting up that data platform is really important because all you’re trying to do as a product person launching a new feature is learn. And, if I have x engineering resources, I will prioritize instrumenting and being able to capture that data point of how people interact with the feature above everything else. Because without that, what was the point?
Ryan: There was no point.
Sandhya: If you didn’t learn anything, what was the point?
Ryan: That’s right.
Sandhya: And I love this, and there are a few growth people who really get it. I think folks who have that experimentation approach, the hypothesis-driven approach, they really get it, and they prioritize instrumentation above everything else.
Ryan: Excellent. You’re adding lots of features and I think Amplitude can be commended for the pace of development. I mean, I think everyone would say, “Wow, Amplitude launched something new”. It’s very frequent. The industry as a whole, and there’s a million marketing automation, and marketing analytics, and product analytics, and different tools. It seems like the features that people are adding are starting to meld together. How do you think at Amplitude about what’s next? Is it Mixpanel ads, the ability to contact users through behavioral cohorts, or you’re starting to see this blurring line in terms of landing page testing with analytics now. Where does Amplitude fit in and how do you differentiate yourself from the masses?
Sandhya: That’s a great question. I think what we don’t want to do is fight feature wars. It’s very easy to call something with the name of a feature. I can launch something and give it a name and that doesn’t mean that people get value out of it. So, we are very focused on what’s the value people are getting out of it? And, our vision is to make sure that product leaders can actually execute on their versions using Amplitude.
Sandhya: So, that’s kind of our goal as a company is anyone who is building product today and is growing a product today, should be able to execute on their version using Amplitude. And you are going to keep supporting all the important use cases they have of how they want to do it, right? So for example, now we are thinking about how do we support product personalization? Because, we already have customers who are driving real time product personalization using Amplitude’s behavioral cohorting API’s. So you have all these examples of customers who are ahead of us and they are thinking of how they can leverage Amplitude and the power of our platform and we are kind of really taking the opportunity to learn from them and start productizing those use cases more and more.
Sandhya: So for us, that is our direction, that is our commitment. We don’t believe in doing kind of “me too” things because somebody else launches something with the same name, because honestly, doing that, but doing it in a way that customers get a lot of value, it really works, it’s very fast, it performs every time, it doesn’t time out, those are hard problems.
Ryan: Right
Sandhya: Giving it a name is the easy problem to solve.
Ryan: How do you think about the difference in product analytics between native mobile and say, web?
Sandhya: That’s a great question. The way we think about the future is really platform agnostic. So we have companies using Amplitude or web or mobile or tvOS, VR headsets, voice, Alexa, and I think the reason why we are seeing so much cross channel success is because the user action based data model translates very well over every platform. So, no matter what platform you are on, the kind of actions that you take in the product are pretty universal. So you can give them a universal taxonomy and make sure that your customer doesn’t feel like they’re on different products when they are using you over Alexa, versus using an iOS app, or trying to use Roku tv on you, right? If you want to give them a uniform experience, then you have to have a uniform classification system for what user actions are being taken.
Sandhya: And, we find that it’s all kind of merging together. Even the way analysts report the industry, it used to be that mobile and web were like two separate industries, that’s no longer true. Most people are thinking about omni-channel, even across hardware and software, especially with voice, that’s a big platform that has finally come of age. Everything is about having an omni-channel version now so we are very platform agnostic and we are going to support every platform there is that is drawing and requires an understanding of user behavior.
Ryan: That’s awesome. Any last thoughts you want to leave us with today?
Sandhya: Yeah, I think the one thought I will leave you with is, a big question that we have for the product community is, how do you make sure that your company can be a product-led company? Right, which is something that every company needs to be right now. And, the big gap that we see there is being able to tie the work you are doing to impact. Being able to say that, this investment, the producting made the sale, is going to drive revenue next year, is going to drive market gap next year. And, the way we see it, there’s no leadership without accountability so you have to hold yourself accountable to metrics that really matter for the future of the company. And I think that’s something that everyone in the product and growth community needs to do a much better job of, right? Really think about “Am I just trying to get the feature shipped or am I holding myself accountable to making the future of this company better?” and how does that all tie together? So, that’s kind of the big problem that Amplitude is working on right now, which keeps us very excited.
Ryan: Well, you’re doing a great job and you guys should keep up the good work.
Sandhya: Thank you.
Ryan: And, we wish you the best. We’re glad to have you. Again, I am Ryan Koonce, the CEO of Mammoth Growth and we were happy to have Sandhya Hegde the director of product at Amplitude.
