Using formula metrics in Experiment Results

• Understand the different kinds of formula metrics supported by the Experiment Results chart

• Understand how Amplitude calculates statistical significance for formula metrics

In an Experiment Results chart, using a formula metric offers you greater flexibility when performing analyses. A formula metric is a metric that consists of:

• at least one event, and
• a mathematical operation through which the events interact with each other.

If you've used custom formulas in Event Segmentation, this should be familiar to you. If not, take a minute to go back and read over that article before proceeding.

Create a formula metric

1. In the Primary Metric module, click + Add Metric and choose a formula from the Formula drop down options.

1. Click + Define single-use metric, then select Formula from the Metric Type dropdown in the modal that appears.

1. Click Select event ... to begin selecting events to include in your formula metric. Repeat this step until you've selected all the events you need.

2. In the Formula box, enter the formula for calculating your formula metric. Click here for a list of formulas Experiment Results supports, or here for an explanation of formula syntax.

3. Add a name for this new formula metric. Click Apply when you're done. The metric now appears in your Experiment Results chart.

You can also view this metric in the object management center.

Supported formula functions

Experiment Results supports the formula functions listed here:

UNIQUES:

Syntax: UNIQUES(event)

• Event: Refers to the event you're interested in. This must be a letter corresponding to an event in the Events Module.

Returns the number of unique users who triggered the event.

TOTALS:

Syntax: TOTALS(event)

• Event: Refers to the event you're interested in. This must be a letter corresponding to an event in the Events Module.

Returns the total number of times users triggered the event.

PROPSUM:

Syntax: PROPSUM(event)

• Event: Refers to the event you're interested in. This must be a letter corresponding to an event in the Events Module.

This function only works when grouping by a numerical property on the event. If grouping by multiple properties, the formula runs the calculation with the first group-by clause.

Returns the sum of the property values you're grouping the specified event by.

PROPAVG:

Syntax: PROPAVG(event)

• Event: Refers to the event you're interested in. This must be a letter corresponding to an event in the Events Module.

This function only works when grouping by a numerical property on the event. If grouping by multiple properties, the formula runs the calculation with the first group-by clause.

Returns the average of the property values you're grouping by. This function is equivalent to `PROPSUM(event)/TOTALS(event)`. Learn more about how Amplitude calculates PROPAVG and PROPSUM in this article

PROPMAX

Syntax: PROPMAX(event)

• Event: Returns the maximum value of the property you're grouping the specified event by. The property must be numeric. If grouping by multiple properties, the calculation uses the first group-by clause.

PROPMIN

Syntax: PROPMIN(event)

• Event: Returns the minimum value of the property you're grouping the specified event by. The property must be numeric. If grouping by multiple properties, the calculation uses the first group-by clause.

CONVERSIONRATE (closed beta):

Syntax: CONVERSIONRATE(array of events, conversion window, latency offset)

• Array of Events: Refer to the events in the funnel.
• Conversion window: Refers to the conversion window of the funnel. The unit is in seconds.
• Latency Offset (optional): Refers to latency in the data ingestion to Amplitude. The unit is in seconds.

Note

This function is supported for Experiment metrics only, and is currently only available to certain customers. To gain access, contact your Amplitude Customer Success Manager.

Returns the conversion rate (< 1) from 1st event to nth event of the array. This function internally uses Funnel Query to get the conversion rate.

CONVERSIONAVG (closed beta):

Syntax: CONVERSIONAVG(array of events, conversion window, latency offset)

• Array of Events: Refer to the events in the funnel.
• Conversion window: Refers to the conversion window of the funnel. The unit is in seconds.
• Latency Offset (optional): Refers to latency in the data ingestion to Amplitude. The unit is in seconds.

Note

This function is supported for Experiment metrics only, and is currently only available to certain customers. To gain access, contact your Amplitude Customer Success Manager.

This function only works when grouping by a numerical property on the last event.

Returns the average value of the given property among all the users who completed the conversion. This function internally uses Funnel Query to get the average value among the converted users.

Formula syntax

In your formulas, refer to events selected in the Events Module by their corresponding letter. The functions and the parameters aren't case-sensitive. You can also perform the following arithmetic operations:

• Parenthesis ()
• Subtraction (-)
• Multiplication (*)
• Division (/)

How Amplitude calculates experiment data for formula metrics

Before getting into how calculations of formula metrics work with experiment data, it’s important to understand the overall Experiment Analysis view, which provides details for your experiment.

For formula metrics, Amplitude computes the results for each function independently to find the mean and variance of each one. It then applies the arithmetic operators to the results of these individual functions.

Imagine you've defined a formula metric as `TOTALS(A) + TOTALS(B)`. Amplitude calculates the variances and means of both components of this metric, as well as the covariance.

If you set X equal to TOTALS(A) and Y equal to TOTALS(B), the following statements hold:

• `V[X]` = Variance of X

• `E[X]` = Mean of X

• `V[Y]` = Variance of Y

• `E[Y]` = Mean of Y

• `Cov[X, Y]` = Covariance of X and Y, assumed to be zero for all mathematical operations.

• Addition: Variance: `V[X + Y] = nV[X] + nV[Y]`
Mean: `E[X + Y] = E[X] + E[Y]`

• Subtraction: Variance: `V[X - Y] = nV[X] + nV[Y]`
Mean: `E[X - Y] = E[X] - E[Y]`

• Multiplication: Variance: `V[X * Y] = n^3 mu_y^2 sigma_x^2 + n^3 sigma_y^2 mu_x^2 + n^2 sigma_x^2 sigma_y^2`
Mean: `E[X * Y] = E[X] * E[Y]`

• Division: Variance: Mean: `E[X / Y] = E[X] / E[Y]`

Once you have the mean and variance of the overall formula metric, you can calculate the confidence interval chart and the p-values.

`Formula / Metric: TOTALS(A) / TOTALS(B)`