Using formula metrics in Experiment Results

In an Experiment Results chart, using a formula metric offers you greater flexibility when performing analyses. A formula metric is a metric that consists of:

• at least one event, and
• a mathematical operation through which the events interact with each other.

If you've used custom formulas in Event Segmentation, this should be familiar to you. If not, take a minute to go back and read over that article before proceeding.

Create a formula metric

To add a formula metric to your Experiment Results chart, follow these steps:

1. In the Primary Metric module, click + Add Metric and choose a formula from the Formula drop down options.

2. Click + Define single-use metric, then select Formula from the Metric Type dropdown in the modal that appears.

3. Click Select event ... to begin selecting events to include in your formula metric. Repeat this step until you've selected all the events you need.

4. In the Formula box, enter the formula for calculating your formula metric. Click here for a list of formulas Experiment Results supports, or here for an explanation of formula syntax.

5. Add a name for this new formula metric. Click Apply when you're done. The metric now appears in your Experiment Results chart.

You can also view this metric in the object management center.

Supported formula functions

Experiment Results supports the formula functions listed here:

UNIQUES

Syntax: UNIQUES(event)

• Event: Refers to the event you're interested in. This must be a letter corresponding to an event in the Events Module.

Returns the number of unique users who triggered the event. 

TOTALS

Syntax: TOTALS(event)

• Event: Refers to the event you're interested in. This must be a letter corresponding to an event in the Events Module.

Returns the total number of times users triggered the event.

PROPSUM

Syntax: PROPSUM(event)

• Event: Refers to the event you're interested in. This must be a letter corresponding to an event in the Events Module.

This function only works when grouping by a numerical property on the event. If grouping by multiple properties, the formula runs the calculation with the first group-by clause.

Returns the sum of the property values you're grouping the specified event by.

PROPAVG

Syntax: PROPAVG(event)

• Event: Refers to the event you're interested in. This must be a letter corresponding to an event in the Events Module.

This function only works when grouping by a numerical property on the event. If grouping by multiple properties, the formula runs the calculation with the first group-by clause.

Returns the average of the property values you're grouping by. This function is the same as PROPSUM(event)/TOTALS(event). Learn more about how Amplitude calculates PROPAVG and PROPSUM in this article

PROPCOUNT

Syntax: PROPMAX(event)

• Event: Refers to the event that interests you. This must be a letter that corresponds to an event in the Events card. The event property must be a number. If grouping by multiple properties, the formula runs the calculation with the first group by clause.

  Returns the number of distinct property values for the property the event is grouped by.

  PROPCOUNT is an estimate of distinct property values. This estimate comes from a HyperLogLog algorithm, and its accuracy depends on amount of data it has to work with. Expect a relative error in the range of 0.1% for less than 12,000 unique values, and up to 0.5% for more than 12,000 unique property values, depending on the cardinality of the property.

PROPMAX

Syntax: PROPMAX(event)

• Event: Returns the maximum value of the property you're grouping the specified event by. The property must be numeric. If grouping by multiple properties, the calculation uses the first group-by clause.

PROPMIN

Syntax: PROPMIN(event)

• Event: Returns the minimum value of the property you're grouping the specified event by. The property must be numeric. If grouping by multiple properties, the calculation uses the first group-by clause.

CONVERSIONRATE (closed beta)

Syntax: CONVERSIONRATE(array of events, conversion window, latency offset)

• Array of Events: Refer to the events in the funnel.
• Conversion window: Refers to the conversion window of the funnel. The unit is in seconds.
• Latency Offset (optional): Refers to latency in the data ingestion to Amplitude. The unit is in seconds.

Returns the conversion rate (< 1) from 1st event to nth event of the array. This function internally uses Funnel Query to get the conversion rate.

CONVERSIONAVG (closed beta)

Syntax: CONVERSIONAVG(array of events, conversion window, latency offset)

• Array of Events: Refer to the events in the funnel.
• Conversion window: Refers to the conversion window of the funnel. The unit is in seconds.
• Latency Offset (optional): Refers to latency in the data ingestion to Amplitude. The unit is in seconds.

Returns the average value of the given property among all the users who completed the conversion. This function internally uses Funnel Query to get the average value among the converted users.

PROPCOUNT

Syntax: PROPCOUNT(event)

• Event: Refers to the event that interests you. This must be a letter that corresponds to an event in the Events card. If grouping by multiple properties, the formula runs the calculation with the first group by clause.

Returns the number of distinct property values for the property the event is grouped by. In this setup, the formula retrieves the number of different departments covering all the items for which details were viewed:

PROPCOUNT is an estimate of distinct property values. This estimate comes from a HyperLogLog algorithm, and its accuracy depends on amount of data it has to work with. Expect a relative error in the range of 0.1% for less than 12,000 unique values, and up to 0.5% for more than 12,000 unique property values, depending on the cardinality of the property.

Formula syntax

In your formulas, refer to events selected in the Events Module by their corresponding letter. The functions and the parameters aren't case-sensitive. You can also perform the following arithmetic operations:

• Parenthesis ()
• Addition (+)
• Subtraction (-)
• Multiplication (*)
• Division (/)

How Amplitude calculates experiment data for formula metrics

Before getting into how calculations of formula metrics work with experiment data, it’s important to understand the Experiment Analysis view, which provides details for your experiment.

For formula metrics, Amplitude computes the results for each function independently to find the mean and variance of each one. It then applies the arithmetic operators to the results of these individual functions.

Imagine you've defined a formula metric as TOTALS(A) + TOTALS(B). Amplitude calculates the variances and means of both components of this metric, as well as the covariance.

If you set X equal to TOTALS(A) and Y equal to TOTALS(B), the following statements hold:

• V[X] = Variance of X

• E[X] = Mean of X

• V[Y] = Variance of Y

• E[Y] = Mean of Y

• Cov[X, Y] = Covariance of X and Y, assumed to be zero for all mathematical operations.

• Addition: Variance: V[X + Y] = nV[X] + nV[Y]
  Mean: E[X + Y] = E[X] + E[Y]

• Subtraction: Variance: V[X - Y] = nV[X] + nV[Y]
  Mean: E[X - Y] = E[X] - E[Y]

• Multiplication: Variance: V[X * Y] = n^3 mu_y^2 sigma_x^2 + n^3 sigma_y^2 mu_x^2 + n^2 sigma_x^2 sigma_y^2
  Mean: E[X * Y] = E[X] * E[Y]

• Division: Variance: Mean: E[X / Y] = E[X] / E[Y]

Once you have the mean and variance of the formula metric, you can calculate the confidence interval chart and the p-values.

Formula / Metric: TOTALS(A) / TOTALS(B)