One of the responsibilities of a Solutions Architect is to ensure that Amplitude customers walk away from the onboarding process with a comprehensive data taxonomy for their products. When I ask customers to explain their process of instrumenting this taxonomy, more often than not, the response I receive is, “We don’t really have a process. What’s your suggestion?”
After discussing best practices for instrumentation with our product and engineering team and reviewing the processes of some of Amplitude’s most successful customers, we identified a few key components that constitute a sustainable instrumentation process.
Here are 5 steps you can incorporate into your workflows to get you started on the path to better analytics instrumentation.
Step 0: Empower your engineers with data.
Before we even get to good instrumentation process, there is a critical step 0 that needs to happen. Without this step, no matter how good your overall process is, your instrumentation just won’t be up to par.
The Product Development teams with the best instrumentation practices involve engineers in their product decisions and user research from day one.
“It’s impossible to trust your data if engineering is not invested in the process.” Ryan Ashcraft, Sr. Product Engineer, Amplitude
How do you encourage your engineering team to become more data-driven and align on a clearly defined instrumentation process? Here are some tactical suggestions:
- Collaborate on taxonomy early on in the development cycle with your engineers.
- Once the data is live in production, share interesting insights you’ve discovered with your engineers or encourage them to explore that data in Amplitude. You can also focus your discussions with engineers on how tracking data will positively impact your business outcomes.
- Suggest your engineers to instrument metrics they care about to pique their interest in data and analytics. For example, here at Amplitude, our engineering team has instrumented our site performance metrics.
Data can empower your engineers to feel more invested in the products that they are working on, quantitatively understand the impact of the features they have built, and guide them during difficult decisions such as which features to deprecate. This will also then enable your team to better maintain an efficient and functional analytics instrumentation process.
Step 1: Define a company-wide standard and guidelines for data naming conventions.
Standardize the syntax all teams and all platforms should use for events, event properties, and user properties. Put this standard into an easily accessible guideline by:
- Linking in your internal company wiki.
- If you have a template for feature specs, then this standard should be linked inside that template.
- Sharing the standard during your product development onboarding process.
If there is no alignment, then tracking may break due to teams instrumenting data with different names. Our Data Taxonomy Playbook is a great resource for naming conventions and how to best structure your taxonomy.
Related Reading: Making Data Validation Suck Less
Step 2: Define success metrics and think about questions before a feature is built.
Before a feature is built, PMs and designers should make sure to define success metrics and consider the questions they want to answer with the collected data.
Success metrics are product metrics that you want to track or see changed as a result of your product change. Some questions you may want to answer with your data could include: What does success mean? How do you know if the product change has been successful? What does failure look like? What funnel conversions do you want to track?
Your feature spec template should contain an area for you to write down these success metrics and questions. Once you gather this information, you can then derive events and properties that would allow you to answer those questions and track those metrics. See our article on best practices on how to set metrics for product launches.
Step 3: Collaborate with your engineers and data governor to finalize taxonomy for a new feature.
Engineers should be involved in the entire process from early on because they can best verify the feasibility and the scope of the data you wish to track for a new feature (e.g. is the value of this event property available at the time of that event?). Your engineers are a great resource for prioritizing which pieces of data are most important as well as understanding what context is available at the time of each user action.
Collaboration is key between engineering and teams that consume the data. In an ideal world, no wall exists between analysts, engineering, QA, and the taxonomy/spec builder. Depending on the size and structure of your company, you might also want to consider creating a data governor role, whose responsibilities would entail owning the structure, cohesion, quality, and maintenance of data.
Your engineers are a great resource for prioritizing which pieces of data are most important.
A helpful tip to ensure all key stakeholders are aligned is to include in your feature spec template a table for the planned events, event properties, and user properties that will accompany each feature.
Related Reading: The Foundation for Great Analytics is a Great Taxonomy
Step 4: Document your taxonomy in a central resource that is easily accessible during the actual code instrumentation.
This is important for ongoing alignment, especially within organizations that contain multiple teams who need access to data. Some commonly used tools by Amplitude customers are:
- Google/Airtable spreadsheet
- Confluence/internal wiki page
- Schema in Amplitude’s Taxonomy add-on
Having a central resource is critical because you can then easily copy the necessary context that is shared between features. For example, if you are making a change to your checkout flow, then you can copy all the “cart” properties that should be tracked on all checkout events.
“Alignment has to happen at the engineering level, it can’t happen at the spreadsheet level”. Tanner McGrath, Head of Growth, Amplitude
If you want to make collaboration with your engineering team as frictionless as possible, however, the optimal and most efficient way might be to build a taxonomy repo in GitHub where event types, event properties, and user properties are defined. Every team can then consume from this repository. This ensures, for example, that Android and iOS will always use the same names for the same data. In addition, team members can easily follow commits to a repo, whereas edits to a spreadsheet are harder to follow. A GitHub repo is also readily accessible to the engineer who is writing the tracking code. Lastly, many tools such as Airtable and Amplitude’s Taxonomy add-on also include helpful APIs that can be leveraged to maintain your taxonomy directly within your codebase.
Step 5: PMs and engineers should work together to validate and QA the data once it has been instrumented.
Similar to your code review process, you should have an instrumentation review process. Taxonomy changes should always be sent to a test project in Amplitude before sending that data to production. The engineer or the PM can then create some sample charts with the newly instrumented data to verify its accuracy.
For best practices on how to validate your event data in Amplitude, check out our video walkthrough.
The Importance of a Good Instrumentation Process
Instrumentation is a small but critical piece of your overall data governance strategy that needs to be built into your product development workflow. Following these five steps will ensure that your whole team, from analyst to engineer, is aligned on the goals you are trying to achieve with data.
