ExperimentJul 31, 2024
Experiment Metric Recommendation Revamp
Improvements to the experiment configuration for setting up experiment goals and choosing metrics, including new metric goal types and relaxed requirements for primary metrics.
Improvements to the experiment configuration for setting up experiment goals and choosing metrics, including new metric goal types and relaxed requirements for primary metrics.
Description: We have made improvements to the experiment configuration for setting up experiment goals and choosing metrics. Users can now set goals for experiments using new metric goal types, allowing for individual analysis of each metric. Additionally, we are relaxing the requirement to set a “primary” metric by offering users the option to opt out of the recommendation features.
Why: Users are forced to use at least one, but also only one primary metric in order to define their goals in Experiment. The current state can block teams that want to track many metrics as a measure of success.
This update is designed to relax that requirement so that customers can have either zero or multiple primary metrics, and its implications on recommendations, notifications, and duration estimation. The main goal is related to customer health and adoption - experiments that do not leverage metrics are basically just feature flags, so giving customers more ways to feel comfortable adding metrics is critical as a prerequisite for delivering value with our product.
When: GA on Monday, August 5th.