Value calculation in Experiment Analysis charts

Knowing how the values in your Experiment Analysis charts are calculated can help you understand what your experiments are really telling you, so you can avoid making potentially costly interpretation errors.

These values are derived from a small selection of inputs and formulas, which are described below.

Inputs

The formulas rely on a straightforward set of inputs:

• E: The number of unique users who have been exposed to the experiment.
• M: The number of unique users who triggered the metric event—in other words, the subset of the users who have been exposed to the experiment. M will always be less than E.
• T: The total number of times the metric event was triggered. A single user can trigger the metric event more than once. Amplitude only counts metric events triggered by users included in E.
• S: The sum of all the metric events' property values.
• A: The sum of the average of all the metric events' property values, per user.
• FM: The number of unique users who triggered the events in the funnel, in the specified order.
• FT: The total number of times all the funnel events are triggered in the specified order.

Formulas

The inputs in the previous section are then plugged into the following formulas:

• Unique conversions: (M/E) * 100
• Event totals: T/E
• Sum of property value: S/E
• Average of property value: A/E
• Funnel conversion, uniques: FM/E
• Funnel conversion, totals: FT/E

Examples

For this example, assume the metric event has a numeric event property VALUE. This table is the chronological log of events coming into Amplitude:

In this example, the number of unique users exposed to the experiment—E in the list of notations above—is four (U1, U2, U3, U4). Of those, the number who triggered the metric event (M in the list) is two (U1 and U2). U5 doesn’t count, as they were not exposed to the experiment.

The metric event was triggered three times, twice by U1 and once by U2. Again, U5 does not count.

The sum of all the metric events' property values is 30, and the sum of their average — ((5 + 10)/2 + (15)/1) = (7.5 + 15) = (U1 Avg + U2 Avg) — is 22.5.

With that information, we can plug these values into each of the formulas listed above:

• Unique conversions = (M/E) * 100 = (2/4) * 100 = 50%
• Event totals = T/E = 3/4 = 0.75
• Sum of property value = S/E = 30/4 = 7.5
• Average of property value = A/E = 22.5/4 = 5.625

Funnel example

For this example, let’s define our funnel as events ME1 and ME2, performed in that order. This table is the chronological log of the events coming into Amplitude:

Here, the number of unique users who triggered the events in the funnel in the given order—defined as FM in the list earlier—is two (U1, U2). U3, U4. and U5 didn’t qualify for the funnel: U3 didn’t trigger ME2, and U4 triggered ME2 out of order; U5 never triggered the exposure event, and thus isn’t included in the experiment at all.

The value of FT—defined as the total number of times all the events in the funnel are triggered in the specified order—here is three. U1 triggered ME1 → ME2 once, while U2 did it twice.

Knowing that, we can plug these values into each of the formulas listed above:

• Funnel conversion, uniques = FM/E = 2/4 = 0.5
• Funnel conversion, totals = FT/E = 3/4 = 0.75