How NerdWallet Harnesses Data to Drive Conversions and Retention and Improve Engineering Metrics
With Amplitude, the personal finance site is helping their engineers find the resources they need, faster than ever.
2x*
Increase in mobile CTRs
-58%
Decrease in load time
5s
To get answers
Founded in 2009, NerdWallet has made finding personalized financial advice as easy as logging online, with a trove of resources for “clarity around all of life’s financial decisions.”
Not long ago, the company needed greater clarity around its own decisions, specifically a shift in its product strategy so it could continue to sustainably grow. That’s when NerdWallet switched to Amplitude.
Why Amplitude? NerdWallet was drawn to the depth of insights it could offer into how NerdWallet products were resonating with users. The partnership has yielded measurable results, including increased efficiency, adoption growth, and faster iteration.
Making the switch to Amplitude
Before Amplitude, NerdWallet used Mixpanel and Redash. But, said Sam Dalton, an Engineering Manager at NerdWallet, Mixpanel’s analytics were too superficial—NerdWallet couldn’t, for instance, drill down on behavioral cohorts. Given it’s inability to deliver the depth of insights needed at Nerdwallet, Mixpanel was only used for a small team and not across the entire organization. Meanwhile, Redash could be time consuming, requiring extra hours in AWS to support the preload. As NerdWallet prepared to shift to a subscription-based model, the company knew it required a solution that helped it to better understand its users’ needs. Amplitude offered the granularity NerdWallet was seeking—and cut down the time it took to get answers from weeks to seconds.
There’s definitely been a change in the culture. Before Amplitude, we’d have meetings where we’d be talking about making a decision about some feature and we’d say, ‘Well, let’s go to analytics, come back in a week and report a decision.’ Now we’re able to say, ‘I can have that in five seconds. Let’s move on.
User behavioral analytics to improve engineering metrics
With Amplitude, NerdWallet has homed in on the back-end decisions that lead to a better user experience. As Sam observed in a NerdWallet blog post: “Data provides a lens into user behavior that can be tied directly to engineering effort. Building software from the ground-up with analytics in mind will enable much richer analysis, unlocking even more insight as the product evolves.”
The company uses Amplitude Segmentation and Cohorts to track and understand the impact of a variety of technical metrics, including how long it takes its app to launch, and how that load time impacts retention. That’s helped the company bring its load to time from 12 to five seconds.
The team also augments its performance monitoring systems with Amplitude, which offers the critical picture of what a user experiences when an issue arises. “It’s a piece of our ecosystem that has that view of whatever the user sees when they encounter a bug or an error,” Sam said.
Using Amplitude’s Segmentation, Pathfinder, and Cohorts, for instance, NerdWallet drilled down into the company’s aggregate error rate to discover a spike in errors among a certain subset of users.
“That was a case of where having that user-centric lens on something caught a pretty big system failure that didn’t bubble up through other channels,” Sam said. “It enabled the product team to say, ‘Hey, something is broken here. Here’s a chart that says on this day, this release went out and there was a massive spike that resulted.’”
NerdWallet also turns to Amplitude to inform its browser support. Looking at usage helps Sam’s team prioritize its work. As he put it, “it helps to push back on work that isn’t necessary.”
Indeed, Sam credited Amplitude with empowering engineers.
Often engineers feel like they don’t have any impact on the product roadmap or an ability to affect change, he said. With Amplitude, I feel like I’m giving them a way into that conversation.
Driving Conversion
In addition to improving the velocity of the engineering team, Amplitude has also been a boon to NerdWallet’s conversion efforts. In examining click through rates in Amplitude’s Funnels, NerdWallet noticed that mobile users were two times less likely to click through than web users. “That sparked a lot of investigation into what’s the best user interface for displaying content on a smaller screen,” Sam said.
NerdWallet used Amplitude to measure the user response in real-time as it experimented with various approaches. “Sometimes you find that what you think is a better design does worse,” Sam observed. “Discovering what actually drives click-through rate is not always what you think it is. Having that really quick turn around to measure that is really helpful. It means we don’t waste a lot of time chasing what we think is a perfect solution only to realize users prefer something else.”
In the end, NerdWallet increased mobile click-through rates by 200 percent.
Boosting retention
Additionally, NerdWallet uses Amplitude to boost retention across it’s product suite. For instance, NerdWallet has used Amplitude’s Cohorts to study the connection between NerdWallet’s various product usage and retention. When the company found users who use at least two NerdWallet products had double the retention rate, it launched a cross-selling initiative.
“Being able to break things down to that level has helped us figure out where we have traction and should be focusing our time,” Sam said.
