Are You Data-driven, Data-informed or Data-inspired?

Have you noticed recently an increase in the usage of the terms ‘data-driven’, ‘data-informed’, and ‘data-inspired’ around your office? It might seem like your co-workers are just jumping on the buzzword bandwagon and throwing words around. But, you would be remiss to completely ignore them because these three phrases hold powerful meaning and are incredibly useful if you know what they represent and how to apply them correctly.

What’s the real value behind these words? What does data-inspired actually mean and how is it different from being data-informed?

Hopefully, I can help shed a little light. In my 10 years working on analytics teams and now on a product team, I’ve been able to understand and illuminate the differences between these phrases in order to use data to strategically guide the product.

First, what do they mean? Data-driven, data-informed, and data-inspired describe how data should be used.

• if you are data-driven, you have the exact data you need to make a decision. If you are being data-driven, you agree with this statement, “It will tell you exactly the answer you need to know in terms of what to do next.”
• data-informed means everyone is aware of the current performance and why the product is performing the way it is in order to make optimizations to your strategies.
• data-inspired means trendspotting. This takes a few different data sources to put the story together since predicting future customer expectations is difficult to do with one data source.

Download the flowchart as a PDF

The best-case scenario is your teams have established the mindsets to leverage all three use cases and are aware of when to leverage each one because:

1. Using these terms properly removes friction between teams because these terms are guiding instructions for a desired output of data
2. They ensure your teams are using the data properly. You don’t want to be caught in a situation of using data-inspired methodologies to answer data-driven questions and vice versa.

Are you data-driven?

Data-driven means that you have the data that will determine the outcome of an outstanding decision. When someone uses this term, it means they are requesting the most specific type of information compared to being data-informed or data-inspired.

This is the most rigid data mindset as the data:

1. Has to be determined upfront via a measurement plan much before the data is needed to ensure the data is properly implemented.
2. Requires a predetermined sample size (typically fairly high) to ensure that the finding is stable enough to replicate.
3. Necessitates the involvement of team members with knowledge of statistical methodologies.

When to be data-driven

The two best use cases for data-driven outcomes are answering business questions and ensuring any changes to a product won’t negatively impact the business.

These are examples of data-driven questions:

• When is the best time to release X or do promotion Y?
• What design performs better, version A or B or N?
• What personalization methodology performs better?
• How much money will we make next month or year, so I can plan my budget?
• How will adding new features/types of users impact the product?

How to leverage Amplitude for data-driven requests

• When running A/B tests ensure that your User Properties are set up to capture those who are in each of the test groups. Upon completion of the test, use the Event Segmentation Report to show which group had higher conversion rates on the success metric dictated from a predetermined measurement plan.
• When running regression models, be sure to use the Compass and Impact Analysis tools.

The Compass tool allows your team to do a very fast exploratory analysis of what actions are highly correlated to each other and highly correlated to the desired outcome. These are the first pieces of information an analyst needs in order to determine what to bring into a regression. From there, the analytics team can easily access the raw data, already formatted in a way they need (columnar data structures) through Amplitude API.

Impact Analysis helps your team quickly identify which features are impacting other features. This can help you spot what you can expect to see in terms of revenue uptick/downtick as a result of changing a feature.

Limitations

With this type of request, the data is meant to answer a very specific question. It means that you can’t source other insights from that same data; the data has a one-dimensional use. If you see others using data from a data-driven request answering a question that it was not intended to answer, that will counteract the effort of being data-driven.

This use case should not guide a new strategy or the design thinking process, but rather it should be used to validate the solution.

Are you data-informed?

Data-informed means that the team understands the performance of KPIs, drop off rates, user pathing within any given product. They are able to spot generalized upticks or downticks in performance and can attribute the reasons why that may have happened. To be a data-informed team you need to know both what and why. The team must also be prepared to use this information to refine and inform their future strategies. Informing a future action with this data will get your team to data-informed.

When to be data-informed

The top reasons to activate a data-informed process are:

1. When strategizing about changes and optimizations to make in a product
2. Prioritizing a backlog of features

To make data-informed work, the team needs to operationalize two processes:

1. Use a hypothesis-driven approach to explain why you think a strategy will work; this informs the why
2. Communicate strategies and release dates to all teams to ensures everyone is informed

How to leverage Amplitude for data-informed requests

Dashboards via Team Spaces will be the single most important way to make sure your team is data-informed. Reports such as Pathfinder, Event Segmentation, Funnel Analysis and Lifecycle should be included in these dashboards. Team Spaces should have the following details:

• Definitions of KPIs and Metrics explaining why that KPI/Metric is being used for this report
• Information on how to interpret the data
• Annotations on what strategies have created impact or changes in these dashboards, so that people can understand trends emerging from exploratory research

Limitations

Unlike a data-driven mindeset, a data-informed analysis should not be telling you directly what to do. It should be helping explain past failures and successes to drive new strategies.

Teams must be able to clearly communicate hypotheses around why the strategies they are crafting will work. If they are not doing this, then the past strategy cannot be analyzed. If the past strategy cannot be analyzed, then the team can never understand why certain things happened and, therefore, never realize a data-informed team.

Are you data-inspired?

Data-inspired generally has no requirements or expectations on outcomes; it’s exploratory in nature. The outcomes of these types of analyses mash up data from different sources and produce interesting commonalities across data sources. The key thing to understand about this analysis is that the person doing it is drawing on intuition and inference as opposed to concrete, statistically sound methodologies. Frankly, that’s a good thing. You want to have this type of analyses in the mix.

The thing with data is that it can only describe what has happened in the past and project what may happen in the future based on past trends. It doesn’t do a good job of coming up with new, innovative ideas because there is no precedent for those new ideas. Data-inspired can help overcome that conundrum of data by identifying seemingly unrelated data sources to inspire new ideas.

When to be data-inspired

The best places to apply a data-inspired mindset are:

1. In a strategy phase that is identifying the innovative direction for a product
2. During Design Thinking exercises

The data won’t ever say how to go about your design or strategy, but it will say that these are some interesting concurrent trends, thus giving more context to the broader picture than the other methodologies.

How to leverage Amplitude for data-inspired requests

Amplitude has made some of the hardest-to-answer exploratory requests into out-of-the-box features standard, so anyone can use them.

Those key exploratory reports are:

1. Personas automatically identify distinct cohorts of users who behave similarly/dis-similarly. Customize the product around each cohort or figure out how to get cohorts that are less engaged to be more engaged. For extra points, sync your cohorts to a Customer Data Platform or A/B testing tool like Braze to customize.
2. Engagement Matrix answers what actions are valuable vs. not valuable to return visits. This report is great at identifying low frequency, but high value actions. Once those are discovered, hypothesize why that action is important to those who do it and how you can facilitate more people finding the same value out of that action.
3. Stickiness shows how often people are coming back and performing valuable actions. Once you know from the engagement matrix low volume, high value actions, use the stickiness report to build custom cohorts of those that perform this action in each visit and compare to cohorts that only performed the action once or not at all.

Limitations

This data should never be referred to as concrete data. Everyone should be aware that the data trends showing up could be spurious interactions (seemingly related interactions but are actually not related). So long as everyone knows it’s a risk drawing concrete conclusions from this data, it’s a risk worth taking because this data sheds light into areas previously clouded in darkness.

What’s your answer, are you data-driven, data-informed, data-inspired or all three?

Leveraging all three data mindsets will grow the amount of relevant insights your teams need to do their jobs. The key is to know when you need to leverage each one so that your expectations of the data are in-line with what you will get from the data.

If you can identify that you are in need of specific data-driven insights, now you know that this may require planning and custom implementations.

If you are looking to be informed of how your strategy performed so that you can optimize for the future, be sure that your teams have been informed of previous strategies and evaluation metrics for those strategies.

If you are looking for inspiration, know that you may not receive the exact information you were hoping for, but may find something that surprises you. All three mindsets have a place, it’s about knowing which ones you need.