Value calculation in Experiment Analysis charts
Understanding how Amplitude calculates the values in your Experiment Analysis charts helps you interpret experiments correctly and avoid costly errors.
Amplitude derives these values from a small set of inputs and formulas.
Inputs
The formulas use the following inputs:
- E: The number of unique users exposed to the experiment.
- M: The number of unique users who triggered the metric event, as a subset of the users exposed to the experiment.
Mis always less thanE. - T: The total number of times users triggered the metric event. A single user can trigger the metric event more than once. Amplitude only counts metric events triggered by users included in
E. - S: The sum of all the metric events' property values.
- A: The sum of the average of all the metric events' property values, per user.
- FM: The number of unique users who triggered the events in the funnel, in the specified order.
- FT: The total number of times all the funnel events fire in the specified order.
- R: The number of unique users who triggered the starting event after exposure, then triggered the return event within the specified time window (nth day, week, or month).
Formulas
Amplitude plugs the inputs into these formulas:
- Unique conversions:
(M/E) * 100. - Event totals:
T/E. - Sum of property value:
S/E. - Average of property value:
A/E. - Funnel conversion, uniques:
FM/E. - Funnel conversion, totals:
FT/E. - Retention:
R/E.
Examples
For this example, assume the metric event has a numeric event property VALUE. This table is the chronological log of events coming into Amplitude:
| User | Event type | Metric event property value |
|---|---|---|
| U1 | Exposure event | |
| U1 | Metric event | 5 |
| U1 | Metric event | 10 |
| U2 | Exposure event | |
| U2 | Metric event | 15 |
| U3 | Exposure event | |
| U3 | Exposure event | |
| U4 | Exposure event | |
| U5 | Metric event | 20 |
The number of unique users exposed to the experiment (E) is four (U1, U2, U3, U4). Of those, the number who triggered the metric event (M) is two (U1 and U2). U5 doesn't count, because U5 wasn't exposed to the experiment.
Users triggered the metric event three times: twice by U1 and once by U2. U5's metric event doesn't count.
The sum of all the metric events' property values is 30. The sum of their average:
((5 + 10)/2 + (15)/1) = (7.5 + 15) = (U1 Avg + U2 Avg) = 22.5
Plug these values into each formula:
- Unique conversions:
(M/E) * 100 = (2/4) * 100 = 50%. - Event total:
T/E = 3/4 = 0.75. - Sum of property value:
S/E = 30/4 = 7.5. - Average of property value:
A/E = 22.5/4 = 5.625.
Funnel example
For this example, define the funnel as events ME1 and ME2, performed in that order. This table is the chronological log of the events coming into Amplitude:
| User | Event type |
|---|---|
| U1 | Exposure event (EE) |
| U1 | Metric event 1 (ME1) |
| U1 | Metric event 2 (ME2) |
| U1 | Metric event 2 |
| U2 | Exposure event |
| U2 | Metric event 1 |
| U2 | Metric event 2 |
| U2 | Metric event 1 |
| U2 | Metric event 2 |
| U3 | Exposure event |
| U3 | Metric event 1 |
| U4 | Exposure event |
| U4 | Metric event 2 |
| U5 | Metric event 1 |
| U5 | Metric event 2 |
The number of unique users who triggered the funnel events in the given order (FM) is two (U1, U2). U3, U4, and U5 didn't qualify for the funnel: U3 didn't trigger ME2, U4 triggered ME2 out of order, and U5 never triggered the exposure event and isn't part of the experiment.
The value of FT, the total number of times users triggered all the funnel events in the specified order, is three. U1 triggered ME1 then ME2 once, and U2 did it twice.
Plug these values into each formula:
- Funnel conversion, uniques:
FM/E = 2/4 = 0.5. - Funnel conversion, totals:
FT/E = 3/4 = 0.75.
Retention example
For this example, define retention as users who trigger a starting event after exposure, then trigger a return event within seven days (Day 7 retention). This table is the chronological log of events coming into Amplitude:
| User | Event type | Timestamp (days after exposure) | Days from starting event |
|---|---|---|---|
| U1 | Exposure event (EE) | 0 | |
| U1 | Starting event (SE) | 1 | |
| U1 | Return event (RE) | 8 | 7 |
| U2 | Exposure event | 0 | |
| U2 | Starting event | 2 | |
| U2 | Return event | 9 | 7 |
| U3 | Exposure event | 0 | |
| U3 | Starting event | 1 | |
| U3 | Return event | 5 | 4 |
| U4 | Exposure event | 0 | |
| U4 | Starting event | 3 | |
| U5 | Exposure event | 0 | |
| U5 | Return event | 6 |
The number of unique users exposed to the experiment (E) is four (U1, U2, U3, U4). U5 doesn't count, because U5 wasn't exposed to the experiment.
For Day 7 retention, Amplitude counts a user as retained if they trigger the return event between seven days and seven days plus 24 hours after the starting event. The number of unique users who triggered the starting event after exposure, then triggered the return event within the Day 7 window (R), is two (U1 and U2):
- U1 triggered the return event 7 days after the starting event (day 8 from exposure, which is 7 days after day 1).
- U2 triggered the return event 7 days after the starting event (day 9 from exposure, which is 7 days after day 2).
- U3 triggered the return event only 4 days after the starting event, which falls outside the Day 7 window.
- U4 triggered the starting event but never triggered the return event.
- U5 never triggered the exposure event and isn't part of the experiment.
Plug these values into the retention formula:
- Retention:
R/E = 2/4 = 0.5 (50%).
Was this helpful?