How to use experimentation to drive acquisition and activation
Once you’ve implemented the strategies and tactics covered in the previous chapters, there’s no reason to stop there. You can continue to improve your acquisition and activation efforts. Start with experimentation, which enables you to explore what resonates with existing and prospective customers, saving time and money.
Experimentation is also a way to continuously refine your knowledge rather than relying on assumptions. It bridges the gap between your perception of what customers need and the reality of what they actually do. This becomes especially important as you scale when it’s more challenging to glean customer feedback.
The following framework showcases how you can experiment with your homepage—one of your most powerful acquisition sources—to drive growth. You can also apply this framework to:
- Marketing campaigns: Test landing page copy or design and measure experiment success with new sign-ups.
- Acquisition channels: Test messaging or sign-up forms and measure CAC and ROAS.
- Onboarding experiences: Test pricing models, customer flows, or notifications and measure the activation rate and time-to-activate.
Seven-step experimentation framework
Experimentation is a continuous process, not a one-time tactic. Following this seven-step framework from Elena Verna, former head of growth at Amplitude, will help ensure your program is aligned with business goals and customer problems and yields actionable insights.
Use this seven-step framework to get started with experimentation. Make each bubble a column in a spreadsheet, then fill it out as you go through each step.
Step one: Define a growth lever.
Meaningful experiments address a business need or problem. In this playbook, we focus on acquisition, but you can also apply this framework to other customer journey stages, such as retention and monetization.
For example, if you notice a high drop-off rate for your homepage, this is how you would frame your experiment: “Accelerating acquisition is our priority, and our highest-trafficked landing page (the homepage) is underperforming.”
Step two: Define the customer problem.
Before you go any further, define the problem the experiment is trying to address from the customer’s perspective. Start by defining an initial customer problem by stating what you think the problem is.
For the homepage example, that might be: “Customers are confused about our value proposition.”
Step three: Develop a hypothesis.
Articulate why you believe the customer problem exists. As with the problem, you’ll iterate on your hypothesis as you learn more, but these first versions give you a starting point for experimentation.
In this case, you might have: “Customers are confused due to poor messaging. Our page has too many action buttons, and our copy is too vague.”
Step four: Ideate possible solutions with KPIs.
Come up with all the possible solutions that could resolve the customer problem. Identify a way to measure the success of each solution by indicating which KPI each solution addresses.
In our homepage example, if one of your solutions is “iterating on the copy,” your KPI might be “improving the conversion rate.”
Step five: Prioritize solutions.
Decide which solutions you should test first by considering three factors: the cost to implement the solution, its impact on the business, and your confidence that it will have an impact. To weed out low-impact and high-cost solutions, prioritize your solutions in the following order:
- Low cost, high impact, high confidence
- Low cost, high impact, lower confidence
- Low cost, lower impact, high confidence
- High cost, high impact, high confidence
The weight you give these factors depends on your company’s lifecycle stage. For instance, if you are well-established with a sizable budget, you may be less cautious about testing high-cost solutions.
Experimentation will help hone your ability to make a confidence assessment. After experimenting, check if the solution had the expected effect and learn from the result.
Step six: Create an experiment statement and run your tests.
Collect the information you gathered in steps one through five to create a statement to frame your experiment. Define a baseline for the metric you’re trying to influence and begin to test.
The statement for our homepage example reads: “Accelerating acquisition is our priority, and our highest trafficked landing page—the homepage—is underperforming because our customers are confused about our value proposition due to poor messaging, so we will iterate on the copy to improve the conversion rate.”
Step seven: Learn from the results and iterate.
Based on the results from your tests, return to step two, update your customer problem and hypothesis, and continue running through the loop. Stop iterating when the business priority—the growth lever—changes or when you see diminishing returns. This might happen for several reasons, including a lack of infrastructure to support your experimentation or resources to effectively solve your customer problems.
Learn how to drive innovation with an experimentation culture in the Guide to Scaling Product-led Experimentation >
Learn how to onboard new users faster with our step-by-step video guide >
In Amplitude, use Experiment to quickly and easily run A/B tests for reliable results. How does it work? Imagine you run a financial services company looking to improve acquisition. Your journeys chart shows that most users who don’t complete onboarding drop off at the personal verification stage.
This leads you to hypothesize that making the personal verification step optional would improve acquisition. To test this hypothesis in Experiment, take the following steps:
Step one: Use metrics from Amplitude Analytics to set your success criteria and roll out an experiment flag that tests your new onboarding design.
Step two: Observe the results as the experiment runs to confirm whether your hypothesis is correct.
Step three: Add the results to your team dashboard to socialize your findings, triggering Slack and email notifications.