Impacts of Analytics Reporting Interfaces on Digital Analytics Programs

Adam Greco

Product Evangelist, Amplitude

people reacted
6 -minute Read,

Posted on November 30, 2021

Does the type of analytics reporting interface matter?

reporting-interfaces

A few months back, I changed worlds completely and went from being an “Adobe Analytics” person to an “Amplitude Analytics” person. In my product evangelist role at Amplitude, I get a chance to speak to a lot of our customers. Some of those customers use Amplitude and Adobe Analytics and some recently shifted over to Amplitude from Adobe Analytics. Since I just came from the Adobe Analytics world and am in the process of learning Amplitude, I have made it a point of talking to customers who use (or used to use) Adobe Analytics. These people speak my language and speaking to them helps me understand the nuances of how the products differ.

After having several conversations with clients who know both tools, one common theme that surfaced was product reporting ease of use. To be honest, the feedback that I heard surprised me a bit. The general summary of the feedback was that the expert analytics teams find the freeform nature of the Adobe Analytics Analysis Workspace interface valuable and easy to comprehend, but that the casual users at the organization found it difficult to use. This prompted me to think more generally about the differences between the reporting interfaces in products like Adobe Analytics (freeform) vs. products like Amplitude (prescriptive). While this post will focus specifically on the differences between Adobe’s Analysis Workspace reporting interface vs. Amplitude’s reporting interface, the concepts shared here will apply to any freeform (like general purpose BI or customer analytics solutions) vs. prescriptive analytics reporting interfaces.

Freeform Reporting

As an expert in freeform reporting products like Adobe’s Analysis Workspace, it was hard for me to hear this feedback because I think freeform reporting is amazing. You can build tables of data in seconds, right-click to do breakdowns, add segments and so on. I love the flexibility of freeform reporting since I often know what data I have, how I want to report on it and so on.

But as I consistently heard the same sentiments from multiple clients, I was reminded of a consulting project I did a few years ago. I was hired by a large financial services firm to train hundreds of casual users on Adobe Analytics, which really meant their Analysis Workspace freeform reporting tool. I created two classes—beginner and advanced, each about four hours in length. But as I started conducting the training, I found that the things I took for granted were really puzzling the class attendees. I watched as they dragged metrics into the freeform table canvas area and then tried to drag dimensions on top of metrics. I saw them drag dimensions onto the canvas and then proceed to ask me what “Occurrences” meant. I was asked why some dimensions were listed twice, but one with a (v) at the end and the other with a (p) at the end (meaning the eVar or sProp version of the dimension). The questions went on and on…

Freeform-Example

After the first few classes, I had to drastically reduce my expectations of what students were going to be able to learn. I decided to just focus on the basics and split my beginner class into two classes, making my new “advanced” class the second portion of my beginner class and only conduct the original advanced class for those who had more Adobe Analytics experience. All the while, I was puzzled because I just assumed everyone would get it. But they didn’t. As is often the case, when you’re too close to something, what comes natural to you can be extremely difficult to the uninitiated.

Prescriptive Reporting

The organizations I spoke to that were now using Amplitude told me that on many occasions, they missed having the cool freeform capabilities provided by the Adobe Analytics Analysis Workspace interface. But they explained to me that the Amplitude reporting interface was easier for their casual users to learn and master. This intrigued me and when I asked why they thought that was, they said that the freeform nature of Analysis Workspace was so unstructured that novice users didn’t know where to start and where to drag items. In some respects, they said it was so powerful that it was intimidating for novices. They mentioned that the Amplitude reporting interface, which uses a more prescriptive model that follows more of a sentence-like structure, enabled users to simply fill in the missing pieces using drop-downs to get the data they needed.

Prescriptive Reporting

 

Prescriptive Interface

Now don’t get me wrong, Amplitude’s prescriptive reporting interface still requires training and a user’s expertise grows overtime, but I could see what they were saying. One customer said to me, “If I were to put a casual user in front of a computer with a freeform reporting interface (like Analysis Workspace) and then have the same user try another computer with a prescriptive reporting interface (like Amplitude), which one do you think would be able to intuitively complete the same data task?” Apparently, they had done that internally and luckily for Amplitude it was the latter.

For organizations that use Adobe Analytics and Amplitude, the people I spoke with said that the core analytics team still uses the freeform reporting interface within Analysis Workspace, but the masses use the prescriptive one in Amplitude. In some cases, usage is split by platform since the desktop website uses Adobe Analytics and the mobile app uses Amplitude. For organizations that have shifted from Adobe Analytics to Amplitude, the people I spoke to sounded altruistic when they said they were willing to give up the freeform interface, which made their lives easier, in order to have a prescriptive reporting interface that worked better for their internal stakeholders.

Unintended Consequences of Ease of Use

While it is easy to say that the digital analytics reporting interface is one detail in the grand scheme of a digital analytics program, some of the conversations I had around this subject with the companies that had used both tools led to some weighty takeaways.

Slower time to insight

A few organizations mentioned that their inability to get infrequent data users to learn the freeform reporting interface led to the creation of a centralized service model for digital analytics. Casual users would submit requests to a centralized analytics team, who would then create reports or dashboards and share them back to the end users. New processes had to be built for accepting and prioritizing analytics requests. The centralized model had inherent time lags associated with it as well.

Lacking context to translate reports into decisions

But what was most troubling for these companies was the fact that the people conducting the analysis were removed from the actual problem that the end user was trying to solve. They didn’t have the full context, which meant that they could only go so deep when creating reports and dashboards. As any analyst knows, when it comes to data, you ask one question, which leads to another question and so on. Analysis is exploratory by nature and when you have someone else doing that exploration for you, you are unlikely to get the true insight needed. So while the central analytics team had an amazing freeform tool in Analysis Workspace that excelled in data exploration, they weren’t the right people to be using it and the right people didn’t understand how to use it (of course, there are many organizations that use freeform reporting interfaces like Analysis Workspace and are able to teach their casual end users how to use it).

Conversely, when they put Amplitude’s prescriptive reporting interface in the hands of the casual users, they were able to run their own reports and continue to explore the data. Since Amplitude grew up in the mobile app/product world, it was common for the same people who built product features to tag them and perform the analysis on their features. Being self-service was an imperative for the product’s target audience. Even though I found the differences between Amplitude’s prescriptive interface and Adobe’s freeform interface to be relatively small, apparently they were enough to avoid the need for a centralized approach to analysis, which in turn created new issues, including negatively impacting results by creating too much distance between the party with questions and the team doing the analysis.

This was a real “aha” moment for me. I always thought of reporting interfaces as similar, but I was shocked how much of an impact having a somewhat easier user interface could have on the entire analytics program. I had heard that it was common for Amplitude customers to have hundreds of people use the product each day (yes, we track that!), but I never considered how the ability to have the masses perform self-service reporting would impact the overall approach to conducting analysis at an organization.

I think it’s important for organizations to think about how they would like to conduct analysis. For some, the centralized model makes sense. For others, a more decentralized, self-service model is a requirement. For large enterprises, a hybrid model may be best. Regardless of which approach your organization chooses, it’s important to be sure that the digital analytics product used is aligned with the model. There’s no right or wrong product, but there may be a mismatch between product and reporting interface that can negatively impact your ability to derive insights from your digital analytics program.


 

Register for AmpliTour

Adam Greco

Adam Greco is one of the leading voices in the digital analytics industry. Over the past 20 years, Adam has advised hundreds of organizations on analytics best practices and has authored over 300 blogs and one book related to analytics. Adam is a frequent speaker at analytics conferences and has served on the board of the Digital Analytics Association.

More from Adam

Inside Amplitude