This article helps you:
Understand the inputs and formulas underlying Experiment Analysis charts
Understand the counting logic Amplitude uses when calculating these values
Knowing how the values in your Experiment Analysis charts are calculated can help you understand what your experiments are really telling you, so you can avoid making potentially costly interpretation errors.
These values are derived from a small selection of inputs and formulas, which are described below.
The formulas rely on a straightforward set of inputs:
The inputs in the previous section are then plugged into the following formulas:
For this example, assume the metric event has a numeric event property VALUE. This table is the chronological log of events coming into Amplitude:
User | Event type | Metric event property value |
U1 | Exposure event | |
U1 | Metric event | 5 |
U1 | Metric event | 10 |
U2 | Exposure event | |
U2 | Metric event | 15 |
U3 | Exposure event | |
U3 | Exposure event | |
U4 | Exposure event | |
U5 | Metric event | 20 |
In this example, the number of unique users exposed to the experiment—E in the list of notations above—is four (U1, U2, U3, U4). Of those, the number who triggered the metric event (M in the list) is two (U1 and U2). U5 doesn’t count, as they were not exposed to the experiment.
The metric event was triggered three times, twice by U1 and once by U2. Again, U5 does not count.
The sum of all the metric events' property values is 30, and the sum of their average — ((5 + 10)/2 + (15)/1) = (7.5 + 15) = (U1 Avg + U2 Avg)
— is 22.5.
With that information, we can plug these values into each of the formulas listed above:
For this example, let’s define our funnel as events ME1 and ME2, performed in that order. This table is the chronological log of the events coming into Amplitude:
User | Event type |
U1 | Exposure event (EE) |
U1 | Metric event 1 (ME1) |
U1 | Metric event 2 (ME2) |
U1 | Metric event 2 |
U2 | Exposure event |
U2 | Metric event 1 |
U2 | Metric event 2 |
U2 | Metric event 1 |
U2 | Metric event 2 |
U3 | Exposure event |
U3 | Metric event 1 |
U4 | Exposure event |
U4 | Metric event 2 |
U5 | Metric event 1 |
U5 | Metric event 2 |
Here, the number of unique users who triggered the events in the funnel in the given order—defined as FM in the list earlier—is two (U1, U2). U3, U4. and U5 didn’t qualify for the funnel: U3 didn’t trigger ME2, and U4 triggered ME2 out of order; U5 never triggered the exposure event, and thus isn’t included in the experiment at all.
The value of FT—defined as the total number of times all the events in the funnel are triggered in the specified order—here is three. U1 triggered ME1 → ME2 once, while U2 did it twice.
Knowing that, we can plug these values into each of the formulas listed above:
Thanks for your feedback!
May 21st, 2024
Need help? Contact Support
Visit Amplitude.com
Have a look at the Amplitude Blog
Learn more at Amplitude Academy
© 2024 Amplitude, Inc. All rights reserved. Amplitude is a registered trademark of Amplitude, Inc.