This article helps you:
Understand the different kinds of formula metrics supported by the Experiment Results chart
Understand how Amplitude calculates statistical significance for formula metrics
In an Experiment Results chart, using a formula metric offers you greater flexibility when performing analyses. A formula metric is a metric that consists of:
If you've used custom formulas in Event Segmentation, this should be familiar to you. If not, take a minute to go back and read over that article before proceeding.
To add a formula metric to your Experiment Results chart, follow these steps:
Click Select event ... to begin selecting events to include in your formula metric. Repeat this step until you've selected all the events you need.
In the Formula box, enter the formula for calculating your formula metric. Click here for a list of formulas Experiment Results supports, or here for an explanation of formula syntax.
Add a name for this new formula metric. Click Apply when you're done. The metric now appears in your Experiment Results chart.
You can also view this metric in the object management center.
Experiment Results supports the formula functions listed here:
UNIQUES:
Syntax: UNIQUES(event)
Returns the number of unique users who triggered the event.
TOTALS:
Syntax: TOTALS(event)
Returns the total number of times users triggered the event.
PROPSUM:
Syntax: PROPSUM(event)
This function only works when grouping by a numerical property on the event. If grouping by multiple properties, the formula runs the calculation with the first group-by clause.
Returns the sum of the property values you're grouping the specified event by.
PROPAVG:
Syntax: PROPAVG(event)
This function only works when grouping by a numerical property on the event. If grouping by multiple properties, the formula runs the calculation with the first group-by clause.
Returns the average of the property values you're grouping by. This function is equivalent to PROPSUM(event)/TOTALS(event)
. Learn more about how Amplitude calculates PROPAVG and PROPSUM in this article
Syntax: PROPMAX(event)
Syntax: PROPMIN(event)
CONVERSIONRATE (closed beta):
Syntax: CONVERSIONRATE(array of events, conversion window, latency offset)
This function is supported for Experiment metrics only, and is currently only available to certain customers. To gain access, contact your Amplitude Customer Success Manager.
Returns the conversion rate (< 1) from 1st event to nth event of the array. This function internally uses Funnel Query to get the conversion rate.
CONVERSIONAVG (closed beta):
Syntax: CONVERSIONAVG(array of events, conversion window, latency offset)
This function is supported for Experiment metrics only, and is currently only available to certain customers. To gain access, contact your Amplitude Customer Success Manager.
This function only works when grouping by a numerical property on the last event.
Returns the average value of the given property among all the users who completed the conversion. This function internally uses Funnel Query to get the average value among the converted users.
In your formulas, refer to events selected in the Events Module by their corresponding letter. The functions and the parameters aren't case-sensitive. You can also perform the following arithmetic operations:
Before getting into how calculations of formula metrics work with experiment data, it’s important to understand the overall Experiment Analysis view, which provides details for your experiment.
For formula metrics, Amplitude computes the results for each function independently to find the mean and variance of each one. It then applies the arithmetic operators to the results of these individual functions.
Imagine you've defined a formula metric as TOTALS(A) + TOTALS(B)
. Amplitude calculates the variances and means of both components of this metric, as well as the covariance.
If you set X equal to TOTALS(A) and Y equal to TOTALS(B), the following statements hold:
V[X]
= Variance of X
E[X]
= Mean of X
V[Y]
= Variance of Y
E[Y]
= Mean of Y
Cov[X, Y]
= Covariance of X and Y, assumed to be zero for all mathematical operations.
Addition:
Variance: V[X + Y] = nV[X] + nV[Y]
Mean: E[X + Y] = E[X] + E[Y]
Subtraction:
Variance: V[X - Y] = nV[X] + nV[Y]
Mean: E[X - Y] = E[X] - E[Y]
Multiplication:
Variance: V[X * Y] = n^3 mu_y^2 sigma_x^2 + n^3 sigma_y^2 mu_x^2 + n^2 sigma_x^2 sigma_y^2
Mean: E[X * Y] = E[X] * E[Y]
Division:
Variance:
Mean: E[X / Y] = E[X] / E[Y]
Once you have the mean and variance of the overall formula metric, you can calculate the confidence interval chart and the p-values.
Formula / Metric: TOTALS(A) / TOTALS(B)
Thanks for your feedback!
July 30th, 2024
Need help? Contact Support
Visit Amplitude.com
Have a look at the Amplitude Blog
Learn more at Amplitude Academy
© 2024 Amplitude, Inc. All rights reserved. Amplitude is a registered trademark of Amplitude, Inc.