DigitalOcean helps developers and growing businesses build and run applications with simpler cloud infrastructure. Our products range from Droplets, our virtual machines, to managed services for AI inference and agents. A customer getting started with us might be creating their first server or trying a new way to build an AI application.
I’m a product manager focused on growth and onboarding at DigitalOcean. I’m responsible for helping customers get from signing up to using our products: finding the right service, adding a payment method, and creating their first resource. I also work with the PMs responsible for our newer AI products to understand what is getting in their customers’ way.
We are helping customers discover and use products for running agents, evaluating models, and serving inference. While we rely on intuition or past benchmarks to get the first draft out, the only way to really improve the experience is to analyze user behavior and set up an experimentation program.
Getting from product analysis to an experiment
Before Wave, our team was already using Amplitude funnels and cohorts to understand how developers moved through onboarding. Click data and session replays helped us investigate where people got stuck. For example, our onboarding funnel showed a large drop at adding a payment method, before customers could create their first resource.
Finding those insights requires substantial work, and converting that to an engineering spec requires even more. We needed to understand the detailed evidence, decide which change was worth trying, talk with engineers about feasibility, and write the requirements for an experiment.
As AI has made development faster, finding the right insights as evidence to decide which product change is worth shipping has become the new constraint.
Haley Eidem, PhD
Product Manager, DigitalOcean
That was the appeal of Wave for me. It could bring customer feedback, replays, and product data together, suggest an improvement, and prepare a plan we could act on. I wanted to see whether its opportunities were worth pursuing and whether engineers could build from its output. I also wanted to know whether we could carry the experiments through to a decision in Amplitude, even as we were still developing the metrics for our AI products.
Trying Wave across five product areas
We joined the beta and set up five product areas: developer onboarding, Harness Runtime, Action Gateway, Evaluations & Simulations, and Inference Engine. That gave us a mix of familiar onboarding problems and questions about newer AI experiences.
Wave brought ranked opportunities with a hypothesis, wireframe, and experiment design. As the product area expert, I could scan them and decide within minutes which were worth pursuing.
I lead our product work with Amplitude alongside Aaron Mitchell, our engineering owner for Platform Experience at DigitalOcean. We bring the product and engineering perspectives together. I review the opportunities and what they could mean for our customers, while Aaron leads the implementation of our experiments. Through our usage of Wave, we also brought in Josh Bailey, Sumant Mehta, and Sathish Jothikumar for opportunities within their product areas.
For the experiments we selected, I used Wave to create the tickets directly in Jira and was able to jump straight to grooming with the team. The engineers built and launched the tests from those files.
Working with them gave us the context of each product as we explored improvements across onboarding and our newer AI experiences. We monitored the tests and reviewed the results in Amplitude, without extra tooling or analyst time. We could also refine metric definitions and statistical settings as we learned.
In only six weeks, Wave surfaced 55 opportunities across those product areas, including four bugs we had not found and four ideas addressing product gaps we had not considered. My time spent writing separate product requirements documents (PRDs) went to almost zero.
Haley Eidem, PhD
Product Manager, DigitalOcean
More customers added a payment method
Our onboarding analysis had already shown that adding a payment method was the biggest point of dropoff. A Wave opportunity proposed making that requirement more visible on the Droplet creation page.
We already had a small indicator within the flow. The experiment moved the prompt into a banner at the top of the page, matching our other creation flows. A banner could help customers notice the requirement sooner, but it could also add clutter. We wanted to test the tradeoff.
Teams that saw the banner were 3.3% more likely to add a payment method than teams in the control group.The result met the test’s configured 80% confidence threshold, and we decided to roll out the banner.
Giving customers control over their dashboard
Another opportunity concerned the Product Spotlight on our home dashboard. We wanted to let customers hide the promotion while watching whether engagement with the products it promoted declined.
Engagement with the promoted products was essentially unchanged between the two groups. We decided to roll out the option, giving customers who did not want the promotion a way to clear it from their dashboard.
Rethinking the Harness Runtime flow
For Harness Runtime, which lets customers create managed agent runtime environments, we tested adding pricing information to the landing page. Our hypothesis was that missing details were preventing customers from creating an environment.
Environment creation was slightly lower with the added information. We could not conclude that the extra information caused a decline, and we chose not to roll out the change.
It did give us another question to test: would a simpler flow with a clearer recommended path help customers get started? That question now informs our thinking across Harness Runtime and Inference Engine.
More opportunities we can act on
Wave gave us a way to take an opportunity through to a built experiment with much less PRD work. I could spend my time reviewing the evidence and deciding what to pursue, while engineering started with a proposed solution and experiment plan.
We plan to roll out the payment banner and spotlight toggle to all customers, then test a simpler Harness Runtime flow. We also have four bugs to fix and four product ideas to scope. As we improve our metrics and close gaps in AI product logging, we expect to keep using Wave to find and test ways to help customers get started.