How MySwimPro uses data to drive growth as a small team

As an engineering team, you don’t just want to ship features. You want to ship impact. And, if you’re in a startup, the quicker the changes, the better.

But to know what your customers want—and what changes to make—data and analytics are crucial.

That’s why MySwimPro made the switch to Amplitude, said the company’s VP of Engineering Nick Newell in a discussion with Erin Del Mundo, Amplitude Success & Renewals Manager.

While the team had the expertise to build its previous, custom solution, it lacked the number of engineers to maintain it.

The move paid off: Amplitude helped the MySwimPro team conduct experiments crucial to its business goals without overextending its resources. The team increased the number of experiments it ran by 10x, and the engineers had more time to work on adding value to the product, work that is critical to the business’s growth.

Speed up experiment velocity

Nick and the rest of the MySwimPro team knew that in order to grow the platform, they had to experiment. The decision to go with Amplitude boiled down to one key factor: the speed of experimentation. With its previous custom stack, the team was only able to run approximately two experiments every quarter. “If you can only run one experiment every couple months . . . you're not going to cut it,” comments Nick.

Now, with Amplitude, the team is able to run 20 experiments every quarter. Additionally, Amplitude has enabled the team to run experiments on multiple platforms simultaneously—including its iOS app, Android app, and web platform.

Nick explains that even if half of the 20 experiments fail, that means ten are successful, “which is huge because each one of those could be bringing 20%, 30%, 50%, or even 70% improvements on your revenue.”

Take risks

MySwimPro is a small but growing business.

“When you're in that position, you have to take some risks,” says Nick, explaining that the team doesn’t wait for their experiments to reach statistical significance.

A smaller pool of daily and monthly active users means waiting for statistical significance would dramatically slow the rate of experimentation.

“We would be back to doing one experiment every two months,” explains Nick. “We wouldn’t be able to make decisions.”

Amplitude gives the team more data to base its decisions on. Amplitude experiments enable MySwimPro’s engineers to know the number of days until their experiment reaches statistical significance. It also gives them extra charts to inform their decisions, like average revenue per user (ARPU).

The ARPU chart shows the lifetime value of a member. When the team runs an experiment, they’ll look at a primary metric, like revenue, then dig into conversion rates and how much revenue is associated with each variant. The team can look at how the data is trending and decide without waiting for statistical significance.

ARPU is just one of the charts the team uses. “There are many tools in the Amplitude platform. . . . [Amplitude] has given us the data that we need to take a little bit more risk,” says Nick.

Focus on individual users to fix product issues

Hearing about a potential user issue without being able to investigate further is frustrating for an engineering team, says Nick.

Being unable to examine individual user journeys, he adds, means “You’re living in the world of the abstract. This is where we were with Google Analytics.”

Now, with Amplitude, the team can examine what happened in the product experience of a specific user for any issues or defects that users report.

“We can look at every single event that has happened to that user prior to when they're experiencing the problem,” says Nick. “It’s transformational.”

As a result, the team has seen improvements in the two DevOps Research and Assessment (DORA) metrics they use to measure quality—change failure rate and mean time to restore. “Those two numbers have gone about 10x in a positive direction since we started using Amplitude,” Nick says.

Move the needle on KPIs

Amplitude gave Nick and his team the extensive user data they needed to understand their customers. The Amplitude platform also made it easy for the team to dig into that data in a targeted way to extract key insights and drive their growth.

Initially, Nick was concerned about the potential data overload that digital analytics tools could generate. However, the team learned that they could use the data to their advantage if they were selective about what they focused on. “You will have a lot of data, but I promise you that you can overcome that with focus and discipline,” advises Nick.

To build that focus and discipline in a small team, Nick recommends focusing only on two key performance indicators (KPIs). The MySwimPro team is currently working on growth and its unique value proposition, and they have one KPI for each area:

• Growth KPI: Revenue
• Value proposition KPI: Number of guided workouts.

Nick shares that Amplitude helps the team access the data they need and ignore the noise. As a result, they’ve been able to prioritize their time on work that pushes their two objectives.

Become data-driven

Amplitude has transformed the culture around data at MySwimPro. “It changes the strategy and the conversation . . . when you have good data,” says Nick. He shares that now, during team meetings, it’s common for someone to raise their hand and say: “Let's just run an experiment, and in two weeks, we're going to know.” The team doesn’t have to argue back and forth. They just go with what the data says.