Letgo relies on Amplitude to keep up with their rigorous A/B testing and now releases features with confidence knowing that they can analyze everything in real time.
A/B tests tracked in Amplitude
The first thing Iago Novoa does when he gets to the office in Barcelona is check his team’s Amplitude dashboards. Iago, the Head of Business Intelligence at letgo, looks at key metrics such as daily active user levels. Amplitude has also become an essential part of letgo’s fast-pace product development process.
Let’s say Iago wanted to analyze an A/B test he ran. Letgo sends the A/B test as a user property to Amplitude and has the ability to analyze anything related to the A/B test. “In Amplitude, it’s just run and drop, anyone can do it. I can create graphs, I can segment, I can do whatever I need,” he said.
“There are so many tools out there which offer analytical options, but none of them come close to the flexibility you have in Amplitude.”
If I didn’t have the option to use Amplitude, he said analysis would be a very time consuming task involving going to their database, building queries and waiting for the query to find the result.
When they need to segment by operating system, or by location, or when they want to check the percentage of users taking a specific action, they send the A/B test into Amplitude as a user property and then can analyze anything they need to know.
“Grabbing the data and making sure you have it all ready to analyze is probably the worst part of analysis for BI people,” Iago added. “Amplitude makes this easy, so we don’t waste any time.”
In addition to A/B test analysis, Amplitude is an integral part of how they monitor the performance of new features at letgo. “Every time we have a new release or we launch a new feature in the app, we wait for a period and run an analysis in Amplitude first,” Iago said. “We thoroughly test everything we do.”
Using Amplitude Insight, their team can also track adoption of new features by setting up certain milestones they want to hit when they first launch. For instance, they might set up an alert when they reach 1,000 users for a new feature. Rather than needing to carefully watch every metric, Amplitude will send an email to a specified list of people who want updates on key KPIs as they are reached.
Amplitude Insight also helps them detect changes in trends related to their critical KPIs. Their team gets detailed anomaly reports that monitor certain performance trends, like if the number of postings drops on a certain day because of a Bank Holiday.
The same goes for detecting bugs and errors. The team gets an alert any time there are errors in a new feature, or if an issue is detected in tracking events or parameters. Iago explains, “we have a bunch of events in Amplitude that fire when there is an error, for example when a user tries to open an item but the item cannot be found. Thanks to Insight, we can quickly detect if the amount of errors has been higher than expected.”
Letgo is a marketplace with tens of millions of monthly active users, so it requires some level of governance to instill feelings of trust amongst their user base. The company aims to make letgo the safest place to buy and sell second hand with the most trustworthy community of users.
To this end, Amplitude helps their Trust & Safety team identify and investigate suspect accounts. Using customer behavior in their product, the team is able to build rich behavioral cohorts that will surface uncommon or problematic behavior in the letgo marketplace. They also use data from Amplitude to guide them in making key product decisions and tracking their progress toward hitting key Trust & Safety OKRs.