Test, Learn, and Iterate Better with Amplitude

As a product manager, it’s critical to understand how your customers use your product—not how you wish they used it. The best way to understand this reality is through data. Without the right data, you can’t move forward.

I joined CloudApp in 2019 as a marketing intern. Today, I’m part of the product management team responsible for self-serve revenue growth. When I first joined CloudApp, we didn’t have a way to challenge or validate our assumptions—and we had a lot of assumptions. CloudApp creates instant business communication through shareable, annotated videos, recordings, and GIFs. We do more than work on the product; we all use the product, too. But that had begun to color our interpretation of how our end customers used CloudApp and how individual users differed from enterprise users.

To move forward, you need to act on reality rather than perception. And you can’t do that without data.

The biggest problem was a lack of data. Many times, someone proposed changing X to make Y better, but those suggestions were based on our assumptions, not fact. We conducted plenty of user interviews, and those offered great qualitative information but didn’t necessarily represent our entire audience. When you have hundreds of thousands of users, it’s impossible to see that larger picture without data. To determine how to enhance our product, we needed to disassociate from our preconceived notions. The best way to do that was to test, and then use those test results to iterate and improve.

A New Solution to Improve Testing

We previously used a customer analytics platform called Woopra, which had product analytics capabilities but no A/B testing functionality and a limited feature set. Then we hired a new lead product manager who had used Amplitude at a previous company. He explained that using Amplitude would allow us to make cohorts of different types of users and track experiments through A/B testing, which we wanted to ramp up. Amplitude was an all-in-one solution: we could drill down into data through A/B testing, but it also offered impact analysis, a pathfinder feature to explore aggregated user flows, and stickiness charts to help us understand the factors driving engagement and retention.

Some platforms require a degree in data science just to get started, but Amplitude is intuitive and easy to learn.

We decided to onboard the platform. After brief formal training, we used CloudApp’s screen recording tool to create a collection of videos that introduced people to Amplitude and taught them the basics. Those videos offered ongoing training opportunities, and people began to explore on their own and learn organically. Some platforms require a degree in data science just to get started, but Amplitude is intuitive and easy to learn.

Cross-Functional Use Cases Lead to Increased Knowledge

Today, all our teams use Amplitude, and it’s become our go-to analytics platform. Our executive team uses it to track high-level metrics. I use it to track experiments while also examining whether those changes affect other parts of the product. Marketing looks at top-of-funnel growth, signups attribution, and personas of those who convert to determine how to best double down on those acquisition areas. Our sales team digs into individual accounts to examine how those accounts use CloudApp features and where to boost our customer value. Meanwhile, our customer success team uses Amplitude to help identify potential issues and bugs that detract from the user experience. Our engineering team also tracks stability and error metrics using Amplitude, especially when we release new features.

Amplitude helps everyone objectively understand where we’re strong and weak. We can identify the areas with the most significant impact and track those impacts. One of the biggest benefits of Amplitude is the ease of sharing. If I want to share a chart I created, I can copy a link to that chart and paste it into Slack for others to see. I usually create a short CloudApp video explaining what I’m seeing that I send along with the chart to help add more context and clarity. Then we can have a broader discussion about what those numbers mean and where we go from there. Amplitude helps everyone see the data for themselves, get on the same page, and start working toward a better reality.

Adopting the Scientific Method to Improve Our Product

In addition to facilitating greater collaboration, Amplitude has helped our overall business goals. We’ve improved retention since adopting Amplitude just by tweaking our activation process.

In our previous flow, users signed up, chose a plan, provided some information about themselves, and finally downloaded the app. Only then could they start to use the product and see its value. We created that flow because we saw something similar with other companies, and we just followed suit. But Amplitude showed us that this flow didn’t engage users as much as we liked. In fact, it got in the way of users activating their accounts. As a result, we shortened the onboarding process significantly, removing the plans and pricing step entirely. Users can now get into the app, try it out, and see the value upfront. This change has improved our conversion rates, and our retention rate has gone up by 51%.

In another experiment, we used Amplitude’s impact charts to examine how engagement with a particular feature impacts other features. By understanding user behavior, we can influence it and encourage users to explore a wider feature set.

Amplitude helps us apply the scientific method to user adoption: creating a hypothesis, testing that hypothesis, analyzing the results, and iterating.

It’s also helped us apply the scientific method to our work: creating a hypothesis, testing that hypothesis through experimentation, analyzing the results, and iterating. We learn from every experiment and build on what we’ve learned to start the cycle anew.

We have a marketing website, a product website, and clients across various platforms. We gather data from these disparate sources and combine it in Amplitude. In my quest to influence self-serve revenue, I use Amplitude to track metrics like conversion rate and the downstream impacts of any change. We don’t want to convert a small set of users right now with a change that will drive away a lot of customers later. Amplitude helps us balance those business metrics in one dashboard and prioritize based on what we see.

Finding a Single Source of Truth

If we didn’t have Amplitude, we likely would have had to find a standalone product designed for experimentation separate from a customer analytics platform. Still, the test-learn-iterate process would have been disjointed without the ability to view those test results within a larger context. With Amplitude, we avoid wasting time going down the wrong path. We make better, more data-informed decisions on the product team specifically, especially when creating new experiences.

Amplitude has become the single source of truth at CloudApp. When you can identify the most engaging touchpoints and areas for improvement, you can give customers the most bang for their buck. Data can change the way you work. Amplitude has given us the capability to harness that data throughout the organization and press forward.

Learn more about Amplitude and Product Analytics by downloading our free Dummies Book today!