How to Get Started with PLG
Learn how to increase product-led acquisition, retention, and monetization with PLG tactics and metrics from over 30 industry experts.
Why Is Experimentation Important for PLG?
While more process than tactic, experimentation plays an outsized role when putting a product-led growth motion into practice.
PLG represents a remarkable opportunity for product and growth teams to directly impact their organizations—and their bottom lines. But with that opportunity comes increased accountability. In a PLG model, product decisions count more than ever.
That’s where experimentation comes in. Grounded in the scientific method, testing allows product teams to explore which new offerings will or won’t resonate with users before rolling them out, saving time and money, and avoiding missteps that could cost them users. The alternatives come with considerably more risk. Even if making product decisions based on intuition or the opinion of the highest-paid person in the room sometimes yields positive results, there’s no way to reliably repeat those results, particularly when you factor in changing markets.
“Our PLG experiments are helping us find the balance between risk and benefit. And it's okay to start small. Small experiments and learning how to track success are great ways to build confidence in the levers for product growth, allowing you to make bigger bets later on.”
Experimentation is also an effective means of promoting a growth mindset that allows you to continuously refine your product knowledge rather than relying on assumptions. It bridges the gap between your perception and the reality of what customers need, which becomes especially important as you scale and it’s harder to glean customer feedback.
Learn how to scale experimentation with Experiment Results.
How NBCUniversal used experimentation to boost retention by 2x
When NBCUniversal launched a new app, it partnered with Amplitude for a complete view of the real-time customer experience. Equipped with this data, it was able to quickly run experiments and release changes to the in-app experience, including the homepage and video preview feature. The result? Its app homepage changes boosted viewership by 10% and increased day-seven retention by 2x. And the video preview feature boosted video-start conversion by 36%.
7-step experimentation framework
Experimentation is only useful when it’s done right. Experiments conducted ad hoc or without the necessary rigor can sink an entire program, causing organizations to abandon the practice altogether. A framework helps by ensuring your experiments benefit your users and, as a result, your business. To keep your experimentation program aligned with your organization’s overall strategy, follow these seven simple steps from Verna’s experimentation framework.
1. Define a growth lever
For an experiment to be meaningful, it needs to matter to the business. Choose an area for your experiment that aligns with the growth lever that is a priority for your business: acquisition, retention, or monetization.
2. Define the customer problem
Define the problem the experiment addresses from a customer perspective. Often, the focus on product-market fit gets lost as businesses grow. To be effective, you need to evolve your understanding of that fit by anchoring your work to customer needs.
3. Develop a hypothesis
Next, define your interpretation of why that problem exists. You’ll iterate on your hypothesis as you learn more. The first version of your customer problem and hypothesis gives you a starting point for experimentation.
4. Ideate possible solutions with KPIs
Come up with all the possible solutions that could resolve the customer problem. Create a way of measuring the success of each solution by indicating which key performance indicator (KPI) each solution addresses.
5. Prioritize solutions
Decide which solutions you should test first by considering three factors: the cost to implement the solution, its impact on the business, and your confidence that it will have an impact.
To weed out low-impact and high-cost solutions, prioritize your solutions in the following order:
- Low cost, high impact, high confidence
- Low cost, high impact, lower confidence
- Low cost, lower impact, high confidence
- High cost, high impact, high confidence
Different companies may attach different weights to these factors. For instance, a well-established organization with a large budget will be less cautious about testing high-cost solutions than a startup with few resources. However, you should always consider the three factors (cost, impact, and confidence of impact). Another benefit of experimentation is that it will help hone your ability to make a confidence assessment. After experimenting, check to see if the solution had the expected impact and learn from the result.
6. Create an experiment statement and run your tests
Collect the information you gathered in steps one through five to create a statement to frame your experiment. Let’s say you’re focusing on acquisition and notice a high drop-off on your homepage. Your statement might look like the following:
Accelerating acquisition is our priority, and our highest trafficked landing page—the homepage—is underperforming [growth lever] because our customers are confused about our value proposition [customer problem] due to poor messaging [hypothesis], so we will iterate on the copy [solution] to improve the visitor conversion rate [KPI].
Define a baseline for the metric you’re trying to influence, get lift, and test.
7. Learn from the results and iterate
Based on the results from your tests, return to step two, update your customer problem and hypothesis, then keep this loop running. Stop iterating when the business priority (growth lever) changes (and set up your experiments around the new lever) or when you see diminishing returns. This might be because you’ve run out of solutions or don’t have the proper infrastructure or enough resources to solve your customer problems effectively.
“PLG is all about rapid experimentation across all funnel stages. Your experiments will fail more than they will succeed, but every learning will pay dividends to help you unlock your growth model. Your model may never be perfect, but you'll continue evolving it to make it better than it was yesterday.”
Experiment brief templates
Good experiment design that adheres to the scientific method can go a long way in ensuring the most trustworthy results. Taking a templated approach makes that easy. It also democratizes the process and creates transparency around your experiment goals and next steps. That’s important because everyone in your organization should be able to run sound experiments and access their outcomes.
Use experiment brief templates for each phase of your experiment, including planning, configuration, monitoring, and analyzing and deciding. The briefs can be categorized by date, user segment, website section, and platform. And most of all, they should be accessible.
Learn how to implement an experimentation program in our Guide to Scaling Product-Led Experimentation.