Run Better Growth Experiments with Qualitative and Quantitative Data

Growth marketing isn’t about quick wins; it’s about building a data-driven foundation that enables you to scale your company indefinitely. At that foundation are your customers—and if you want your foundation to be strong, you need to invest first and foremost in satisfying your customers.

To successfully grow your company, you need to understand what drives your customers to take actions (or not take actions) on your site. This requires a complete view of your customers, their decision-making process, and their journey. You can only get this complete view if you have quantitative and qualitative data. Quantitative data stems from analytics and provides the “what” (the actions they’re taking), where qualitative data comes from user research and provides the “why” (their reason for taking those actions).

Combining rich product analytics from Amplitude with continuous research from a tool like Sprig is the best way to get a complete picture of your sales funnel so you can make the best growth marketing decisions.

Use product analytics to identify customer problems

Product analytics helps identify customer issues by tracking and revealing quantitative data. Quantitative data, like retention rates, bounce rates, and sign-up rates, can indicate problems in your marketing efforts if there are inordinately high or low occurrences of these rates.

Quantitative data tells you what the problem is, whether that’s low retention or high bounce rates. Without quantitative data, any idea you have about what your marketing problem might be is just a guess.

To get quantitative data, you need to run product analytics—the kind Amplitude provides.

Identify the problem

To properly utilize product analytics, you must first with Amplitude to find improvement opportunities like overall customer lifetime value and top-performing acquisition campaigns. This high-level information highlights glaring red flags such as sharp drop-off points with customers on certain days or underperforming campaigns, information that is valuable to both growth marketers and product teams.

Let’s say, when looking at your funnel, you’ve found user count spiked following a campaign but fell shortly after. By the month’s end, you had a handful more users than the month before, but nothing like the beginning of the month. Upon closer inspection of your n-day retention rate, you notice many of your new sign-ups left by day five. Giving you a specific problem to fix: people dropping off after five days with your app.

Get a closer look at your audience

Next, go beyond typical analytics—page traffic and bounce rate—and, or audience segments. Unlike general traffic information, cohorts can be broken into behavioral groups. Making cohort information more meaningful than general traffic, as the cohorts are tied to real-life behavior and actions.

Your high-level analysis showed a sharp customer drop-off on day five of using your product, so you dig into cohorts and see who is actually dropping off. After looking at your n-day retention rate and cohorts who left by day five, you realize many of these customers weren’t using the social sharing feature in your app. Now your problem of people leaving after five days has a thread to pull at: driving the use of the social sharing feature.

Set conversion drivers for the problem

Once you’ve identified the potential fix for the problem you’re solving, set conversion drivers that counter or solve that problem. is an ideal action you want to monitor, like using social sharing in your app or creating a playlist. Conversion drivers vary from cohort to cohort and campaign to campaign. Think: What is it that I want this particular group to do?

You know users who use social sharing tend to stay longer, so social sharing is your conversion driver for this campaign. Use Amplitude to create a conversion driver for social sharing and then move on to hypothesizing how you might drive more customers to try this feature.

Hypothesize and A/B test

At this point, you know what it is you want to fix, and your conversion drivers are set. Now, you need to brainstorm ways to boost your conversion driver and test those hypotheses. Ask yourself: How can we make these conversions a reality?

If you want people to use the social sharing feature to see if these customers stick around for longer than five days you can various app layouts that emphasize the social feature, such as a popup tutorial that calls it out or run an ad campaign that showcases the perks of the social sharing feature. From there, closely monitor your conversion driver performance and see which changes have the biggest impact fine-tuning whatever works the best.

It’s worth noting that you need to capture as much data as possible while running these A/B tests — a failed test can provide as much useful data, if not more, than a successful test.

However, this quantitative data reveals only part of the picture. Qualitative data provides the finer details and allows you to ensure your quantitative-driven changes are made with as much information as possible.

Run user research to delve into the why behind the customer problem

After identifying the issues within your customer funnel, the next step to improving growth marketing is pinpointing why that’s the problem. And you can only identify why your problem exists with qualitative data—data provided through firsthand surveys.

Surveys asking your audience directly about a specific problem or growth experiment, like a newly changed call-to-action or button placement, give you a single source of truth: your customer. This takes away any doubt general analytics may leave and allows you to address the problem directly. But remember: quantitative data provides the information necessary to ask the right questions with qualitative research.

So if quantitative data indicated your problem was a low retention rate after day five of user sign-ups, then you could use qualitative data to help identify why users are leaving after five days.

One of the best ways to gather qualitative data is with microsurveys, a quick survey embedded into your site or app, Sprig provides. From there, you can ask customers to provide feedback about a feature — again, removing any doubt left by your data.

Let’s return to the cohort of people not using social sharing. Using quantitative analytics, we uncovered the correlation between people leaving the app early on and not using the sharing feature. We even tried out various layouts and popups to try driving the use of that feature—all of this possible only because we have quantitative analytics from Amplitude.

Next, using Sprig, we could utilize a microsurvey to ask people why they use—or don’t use—the social sharing feature. We could even ask people about the new placement of the feature or popup being used to promote it to determine why these changes were or weren’t meaningful. This allows you to get to the heart of the issue and make the feature more appealing to a wider audience, thus improving retention rates.

The usefulness of qualitative research goes beyond individual use cases like the one above. can help you uncover your why while you’re still building the hypothesis, as well as helping you to test your hypothesis, allowing you to try out solutions to problems before they’re actually problems.

For example, you can use a to test out various landing pages before they’re even live to see what pages perform the best or worst, and why. Feedback from these micro surveys will help you build the most effective site or app possible before it goes live, so you are making the best first impression.

Using Sprig’s, you can quickly sift through any surveys, whether they’re focused on sign-up barriers or improving a live feature. Sprig’s text analysis will pull key terms, sentiment, and themes from the answers to help you find common ground or recurring issues. With this information, you can quickly make meaningful improvements that align with your customers’ wants and needs.

Create better growth experiments with quantitative and qualitative data

With both quantitative and qualitative data, you now have the answers to how, what, when, where, and why, enabling you to make informed decisions with your UX experiments, solve customer problems, and better your overall understanding of your audience.

When creating your hypothesis for experimentation, leverage the full breadth of data you now have: ideate and test solutions that meet the underlying customer problems vs. just addressing surface-level business metrics.  make this process as painless and seamless as possible. But don’t take our word for it:

Whether your site or app is live or in early testing, your customers are your source of truth. Find that source with quantitative and qualitative data and create an app that makes the best first impression imaginable.

Learn how Opendoor sped up experimentation and drove product growth by combining qualitative and quantitative analytics with Sprig and Amplitude. .