Investing in a new product is a risk. How do you prove it will be successful? How do you measure ‘success’ in the first place? These are just some of the questions the Workplace Analytics team has grappled with as it establishes itself within the Office 365 brand.
MyAnalytics: a Productivity Tool Within Office 365
MyAnalytics and Workplace Analytics are a family of new products within the Microsoft Office 365 offering to help information workers spend their time more effectively. MyAnalytics provides experiences that are embedded into Outlook, a dedicated web app, and a weekly email that is sent to users. The team needed to understand how to effectively engage its users across multiple surfaces to help its users be more productive.
How Microsoft Uses Amplitude
Using Amplitude, the lean team within Microsoft was able to see that their users have a varying frequency of engagement with each surface. The team defines engagement as a user completing tasks that help improve their own productivity such as booking time on their calendar to get things done, declining unnecessary meetings or catching up on missed important mails.
With MyAnalytics, Outlook users can see who they’re spending time with and if it matches their goals
They used Amplitude to quickly identify personas who engaged in a particular set of surfaces more than others. Using this data, the team made strategic product changes to promote more engaging features. It has helped them build more habitual use to increase retention and impact.
View this Customer Story as a PDF
Amplitude allowed the team to enable deeper and faster explorations into A/B testing results of feature changes that were geared towards increasing engagement and impact.
An example of a feature change Microsoft tested is in the simplification of user experiences. The team removed barriers like first run experiences and less important interactions to get more users to easily complete important tasks that increased retention such as exploring insights or booking time to focus.
Impact
Top level view of different actions users take within MyAnalytics.
As a result of this work, the MyAnalytics team saw great results:
- 4x increase in user engagement with MyAnalytics
- Time spent in product is up 4x
- These significant increases in engagement translate to more support and recognition for the incubation product within Microsoft
The team has learned a lot since using Amplitude. Amplitude allowed the team to not only improve engagement, but also to measure and correlate productivity gains with the engagement of new features. The team is constantly discovering better ways of engaging users and have re-prioritized features that were more effective in changing user behaviors or saving users time.
It has also learned that users are not looking for new apps. They want a more streamlined experience in existing apps to help them improve their productivity.
“You will hear criticisms from folks that say, ‘oh, this feature is useless,’” says Gooi Chungheong, a Principal Engineering Manager at Microsoft on the Workplace Analytics team. “Then you look at Amplitude, you can see that’s actually a group of people that are deeply engaged and have saved a few hours each week as a result of the feature.”
In an Amplitude Cohort report, Gooi found that a segment of users who was highly engaged with the product. “They exist!” he said. “It’s a smaller population than what we want it to be but that’s the first step, realizing there’s something here.”
He and his team are working to extend the value from that segment to a much larger population of users. To do that, MyAnalytics is reimagining how to build a more streamlined experience that is integrated into the most popular Office apps, and experiment with the ability of insights and solutions in changing user’s behaviors and improving productivity.
