CHALLENGE

MyAnalytics and Workplace Analytics are a family of new products within the Microsoft Office 365 offering to help information workers spend their time more effectively.

MyAnalytics provides experiences that are embedded
into Outlook, a dedicated web app, and a weekly email that is sent to users. The team needed to understand how to effectively engage its users across multiple surfaces to help its users be more productive.

SOLUTION

Using Amplitude, the lean team within Microsoft was able to see that their users have a varying frequency of engagement with each surface. They used Amplitude to quickly identify personas who engaged in a particular set of surfaces more than others. Using this data, the team made strategic product changes to promote more engaging features. It has helped them build more habitual use to increase retention and impact.

“Amplitude has saved us months of engineering investments into understanding user behavior in our products. This platform greatly reduces the barrier to data analysis.”
-Gooi Chungheong, Principal Engineering Manager, Microsoft

Amplitude allowed the team to enable deeper and faster explorations into A/B testing results of feature changes geared towards increasing engagement and impact.

RESULTS

  • 4x increase in user engagement with MyAnalytics
  • Time spent in product is up 4x
  • These significant increases in engagement translate to more support and recognition for the incubation product within Microsoft

WHY AMPLITUDE

  • Before Amplitude, analysis was slow because data needed to be pre-aggregated into data warehouses for existing tools like Tableau, Excel or Power BI to work. Pre-aggregation slowed down analysis and was often a waste of time.
  • Democratizing data was difficult. Most people who owned product decisions were not data engineers and wanted to focus on answering questions instead of writing queries or ramping upon data in cubes and databases.
  • Hands-on support through onboarding into Amplitude. Amplitude worked closely with Microsoft to ensure the team was setup to be successful.

Proving A New Product Can Help Customers

Investing in a new product is a risk. How do you prove it will be successful? How do you measure ‘success’ in the first place? These are just some of the questions the Workplace Analytics team has grappled with as it establishes itself within the Office 365 brand.

The team has learned a lot since using Amplitude Analytics. Amplitude allowed the team to not only improve engagement, but also to measure and correlate productivity gains with the engagement of new features. The team is constantly discovering better ways of engaging users and have re-prioritized features that were more effective in changing user behaviors or saving users time.

It has also learned that users are not looking for new apps; but needs a more streamlined experience in existing apps to help them improve their productivity.
my-analytics-Product-imageMyAnalytics in Outlook gives users statistics about how they work.

“You will hear criticisms from folks that say, ‘oh, this is useless.’ Sometimes you might feel like there’s no value in data at all,” says Gooi Chungheong, a Principal Engineering Manager at Microsoft on the Workplace Analytics team. “Then you look at Amplitude, you can see that’s actually a group of people that are deeply engaged and have saved a few hours each week as a
result of the feature.”

In an Amplitude Cohort report, Gooi found a segment of users who was highly engaged. “They exist!” he said, reliving the discovery of the cohort.

“Amplitude has been great in terms of pointing out areas where things are not working as well as we expected them to work.”
-Gooi Chungheong, Principal Engineering Manager, Microsoft

“It’s a smaller population than what we want it to be but that’s the first step, realizing there’s something here.”

He and his team are working to extend the value from that segment to a much larger population of users. To do that, MyAnalytics is reimagining how to build a more streamlined experience that is integrated into the most popular Office apps, and experiment with the ability of insights and solutions in changing user’s behaviors and improving productivity.