How I use Wave to find Amplitude's next product improvement

In one of our analyses of Amplitude users, we watched a replay where someone spent 76 seconds writing a question on our Agent homepage. On the same page, probably in the user’s field of vision, our product showed three suggestion cards. Essentially, we took three tries at predicting what that person would want to do, and we missed on all of them. It cost them more than a minute of explaining.

There was a clear mismatch between what we were trying to build and how the actual experience was working. We saw a chance to investigate and improve our suggestions based on each user’s recent work. This is exactly the type of problem to answer with Wave.

We built Wave to answer hard questions about how people use digital products. We use it on our own product too. Across Amplitude, our teams have already used Wave to save text answers, clarify setup requirements, change how agents investigate a request, and fix friction in docs search.

When we first introduced Wave, we followed a bug that was deleting sessions when a user clicked during an agent response. Wave connected replay evidence to customer feedback and helped turn the investigation into a coding agent’s pull request. Our team reviewed and merged the fix.

It’s a useful workflow: use replays to form a hypothesis and analytics to check how widely the pattern occurs. Wave brings that investigation into an Opportunity, including a proposed change and a way to judge it. We can inspect the evidence, revise the plan, and hand implementation to a teammate or coding agent. After the change ships, we return to the same context to see what happened.

Here are some ways we’re using Wave to make Amplitude better.

Making the first question more relevant

Let’s go back to the user who typed past our suggestion cards. That person was part of a Wave analysis of 100 replays. In that sample, users manually typed questions in 80 sessions, and only 12 selected a card. The suggestions were rendering correctly. Their prompts were just so generic that they missed the work people needed to do.

Someone arriving from a dashboard or returning to an existing chart gave us context we could use. Wave’s recommendation was to generate suggestions from that person’s recent chart and dashboard activity. Our team updated the experience to show personalized cards to users with existing context and display generic suggestions to people without that history. All users still had an option to ignore the cards and type their own question.

The original experiment (run this July) reported 2.43 suggestion sends per 100 exposed users for personalized cards, compared with 0.94 for generic cards. That boost (160% more) is only an initial test readout that still requires further validation, but it’s a sign of progress.

Giving useful answers a next step

Once Amplitude helps a user answer a question, they may want to repeat that analysis. We built Custom Agents for exactly that type of recurring work, but there was no clear path to turn a useful response into a Custom Agent. Wave identified that Opportunity for us. It recommended that we add a contextual suggestion that leads into Custom Agent creation when a response represents repeatable work.

We also added chat source attribution to distinguish creation clicks that came from an answer. The four-week readout recorded about 70 unique users a week clicking through the new chat entry point, roughly 8.3% of creation clicks. We can now see people taking that route into repeatable work.

Another part of our evolution was the realization that some answers are worth keeping even when the user doesn’t want to turn them into automated tasks. People could always save a chart, but a structured text explanation had no equivalent action.

Wave proposed new mechanisms to save the results to a Notebook or export/copy them. Our team approved and shipped. Now, saved notebook content includes the answer’s citations and a link to the original response. The closing readout recorded 152 save events over 30 days.

It was working, but that step forward also introduced a new step back: Session Value Rate (a metric we created to judge user experiences) excluded the new save event. The save action was being used, but we needed to update our metric definition before it could accurately evaluate that behavior.

A separate Opportunity extended access to synthesized specialist agent outputs through Model Context Protocol resources. Our team shipped that surface so MCP clients could retrieve those outputs. Now, saving an answer, turning repeatable work into an agent, and exposing outputs through MCP are all ways to use existing agent work in new places.

Helping people reach a working setup

Another Amplitude product, AI Feedback, searches through all the different channels customers communicate about products (reviews, surveys, support tickets, etc.) and consolidates their suggestions. It needs connections to those feedback sources to produce meaningful results, but its empty state wasn’t giving users enough help to make that first connection.

Wave identified the gap, made a recommendation, and our team shipped a new experience with clearer guidance on making those connections. From June 30 through July 29, that Opportunity’s measurement record reported 121 unique users starting setup and 84 completing a source connection, a completion rate of 69.4%.

The next step in our investigation followed people into their connector gallery. We found that people could open a connector and run into a blocker requiring administrator permissions to take the next step. Our team added requirement badges to Intercom and Zendesk when no sources were connected, moving the prerequisite to where people made the choice. We’re tracking completed connections as the metric to determine success in this experiment, and we’re still waiting on the data that will establish an increase.

Another team used Wave to examine data source selection during onboarding. The shipped treatment added recommendation labels for CSV or HTTP API while leaving the tile order unchanged. The HTTP API recommendation rolled out on October 27.

In the direct experiment readout, developer setup activation was 24.61% in the HTTP API treatment and 17.75% in control. The result was directional and statistically inconclusive, so the recommendation remains a hypothesis to evaluate.

Wave also surfaced a guided onboarding checklist proposal. After a new user selected their role, it would give them a recommended action based on that input. That Opportunity has been reviewed, with implementation and impact measurement still unconfirmed.

Changing how the agent approaches the work

Wave identified a Setup Agent flow that was conducting broad scans of data before asking people what they wanted. Our team changed the flow to start the process by establishing the user’s intent. With that information, the Setup Agent can conduct its investigation, offer fast and well-researched paths, and produce shorter proposals.

A Web Vitals investigator had a different problem: it was producing repeated, overlapping performance reports during a run. The team shipped a tighter prompt contract to address that duplication, changing the instructions that govern what the agent produces. Both changes address behavior inside the agent.

For iterations like these, our team reviews Wave’s proposed implementation and decides what reaches users.

Following the friction into docs and the blog

Our documentation search could fetch results while someone was still typing the first characters of a query. A brief, incomplete input could trigger a request and a search event started before the person finished entering a word.

The fix we shipped introduced a configurable minimum of three characters and roughly 200 milliseconds of debounce. Shorter inputs now show a prompt without fetching results or starting a search. The change solves for premature requests and keeps incomplete inputs out of the search telemetry.

On the Amplitude blog, Wave surfaced an Opportunity to use the topic of a post to guide the call to action. The team shipped that targeting through the existing Sanity inline ad component, tailoring the next action specific to what someone was reading. We still need attribution that separates the effect of the topic matching from changes in traffic mix and broader form activity.

Keeping the evidence with the change

With Wave, I can review a proposed interface change alongside a prompt contract or an instrumentation fix, with the evidence that led to it. Our team decides which proposal to pursue and reviews what ships. When the result is unclear, we can return to the original question and identify what the next measurement needs to resolve.

The saved answer feature gives us a concrete next task: include the new save action in the metric definition, then evaluate the behavior it was designed to support. The investigation, the implementation, and the result stay connected in the Opportunity.

Bring a product area and a goal to Wave, and review the opportunities with your team.