Amplitude AI Builders: Janaki Vivrekar Discusses Automated Insights

This is the first post in our Amplitude AI Builder series. Each one will feature an Amplitude engineer discussing an AI product that they are building. They’ll cover what the product does and how they built it.

Data analysts use Amplitude to find answers. They methodically move from data to charts to questions to follow-up questions and eventually to actionable knowledge about their business. That analysis process isn’t straightforward. It’s a complex circuit of dead ends and discoveries. It’s extremely valuable, but often time-consuming and unpredictable.

For this post, I talked to Janaki Vivrekar (senior software engineer at Amplitude) about her team’s ambitious attempt to investigate and recreate the data analysis process with agentic AI. Our conversation covers the one sentence that kicked off her project, the steps her team took to dissect data analysis, and the future of AI analytics at Amplitude.

How are you using AI to find faster insights in Amplitude?

We built an agentic AI system to analyze data in Amplitude and present insights to users. We’re trying to stay true to how analysis happens today in Amplitude, just using AI to automate and accelerate that process.

As a company, Amplitude has two main strengths: 1) we have so much data from our customers, and 2) we know how people do analytics. So we're putting those two things together to create something that still generates valuable knowledge from data, only without all the manual steps and complexity.

What was the inspiration behind automating insights in Amplitude?

It started in a meeting with our CEO Spenser. He told us that we should become the world's experts on what an insight really is, and then automate that process for our users. We took that and we ran with it.

Our automated insights project hits right at the heart of Amplitude's mission. Even before AI was possible, one of the things we focused on was giving customers insights about their data as fast as possible. To take the next step, we needed to dig into the way analysts move from data to knowledge.

That’s the spirit of this project. We want to do more than just present data and charts to our customers. That access is just a stepping stone. We want to get them to the end goal—faster realizations of the insights buried in their data.

So your team has an important but abstract question to research. How did you start?

We have so much internal data, and also data from hand-raiser customers who are really excited about the potential of AI for product analytics. We wanted to use that data to uncover insights the way a human does, and then model that process inside an agentic AI system.

So we dug into what a data analyst thinks when they sit down to do analysis. When they see a bump in the data and they start sorting through what could have caused it. How do they process information? How do they segment the data to understand what aspects could be causing that spike? Then, how do they draw conclusions? In a nutshell, that's what we're trying to build: an agentic AI system that can conduct root cause analysis.

We didn't know if the AI could even look at data and identify interesting insights, so we tested the technology on our internal charts. We had a bunch of data that real product managers had to spend hours analyzing during past product launches. We ran those charts through our system and discovered that our AI was able to do what real analysts were doing, only a lot faster. It could identify spikes or dips in metrics and make correlations to events like product releases and launches. That's when we realized we were onto something.

What did you learn about the analysis process when you dug into it?

Analysts look for relevant properties to use to segment data. Then they use those segments to modify the chart and see if anything jumps out. For example, maybe segmenting by country shows that one individual country is contributing to an increase in the data because of a marketing launch specific to that geography.

Our agentic AI system mirrors that process. It executes multiple different chained tool calls, thinking like an analyst would think. It looks at the chart data first. Then searches across experiments and product releases from that same time period. It pools together business context from past annotations that users have made on the chart. It finds other dashboards, notebooks, and cohorts that could be related. Once it has this context, it reasons about relevant properties, segments, and notable trends or anomalies in the data.

Our agentic insights investigator can run through these explanatory loops all at once. Then it summarizes all of its findings and starts to speculate about why things happened.

AI is great at this kind of work. If a human had to read through 70 different product releases, 50 experiments, and 20 annotations, then do 100 different segmentations to identify what is useful, that would take a lot of time. An agentic AI system will produce the same quality of work as an expert analyst, but it can do it in a matter of seconds or minutes. It’s a giant time saver, and you don’t have to be a data expert to use it.

The core of this project is turning the art of data analysis into a repeatable, optimizable process that an agentic AI system can replicate. How did your team make sure your technology was taking the right steps?

We started by creating a set of evals. We built out an entire sheet to map analytics questions to steps that humans actually performed as they investigated their chart data. On top of that, we built a homegrown taxonomy so we could define and understand the categories of analytics questions users are asking and what types of insights they tend to seek.

We charted common steps and cycles that resulted in successful explanations. For example, we found that a common cause for data anomalies is seasonality. Maybe there’s a holiday causing a dip in metrics. Maybe there’s a product release that drove growth. Maybe there’s identifiable bot behavior that’s impacting the data. Every time we discovered a new analytics pattern, we logged it in our sheet and augmented our set of evals to reflect this.

Then we iterated on how the system performs analysis and tracked the results. We tracked failures and successes. We even saw some cases where it partially gets the insight. Every time we see a miss, we ask: “Why did it fail? Why was it not able to find this insight that a human was able to? What capabilities is the system lacking?” Eventually, we even automated the generation of some of our evals, using narrative reports of analyses in existing Amplitude notebooks.

What do you imagine for the future of automated insights in Amplitude?

We want this to be a system that helps users do what they've already been doing, just faster. It’s not some magical AI system that's going to promise everything. We want to get this technology to customers by the end of the year so we can watch how they use it and see how it changes their data analysis.

We’re experimenting with different approaches that use an initial prompt from the user to point it in a certain direction. Maybe there's a future where we expose this functionality through an LLM chat interface. Or maybe this runs automatically in the background on important charts, and then proactively pings users with insights that they weren't even actively looking for.

Right now, we’re sitting between descriptive analytics (explaining what is happening) and diagnostic analytics (determining why it happened). In the future, maybe we can build an engine for predictive analytics.

What excites me the most about this work is that it’s a way to use new tools to build something that’s genuinely unprecedented. It’s not just marginal tweaks to improve efficiency or usability, it’s an entirely new way of approaching deep research for analytics. Something like this wasn’t possible a year ago. But new AI tools are paving the way to a future where customers can use Amplitude to execute the traditional data → insights → action loop in minutes rather than months.