This article covers frequently-asked questions about Amplitude's Experiment Results chart.
A common mistake is to attempt to generate a chart using only one variant. The Experiment Results chart needs something to compare your control to in order to generate results. If you neglect to include both the control and at least one variant, your chart will not display anything.
Why is my graph displaying an error state?
When using a T-test, you will have to wait until your experiment reaches the specified sample size before Experiment Results will run the p-value and confidence interval computations. With sequential testing, even with a large MDE, it can take some time to reach statistical significance if your experiment’s lift is small. A T-test will generally require fewer samples to detect the same lift.
Why is reaching significance taking longer than it should?
Amplitude uses two parameters to calculate Return On for the Retention metric: For example, a user performs an exposure event and a return event: With a return on the nth day value of seven (seven days), the user will be counted as retained if the If the return on value shifts to an nth week value of one (one week or seven days), the user will be counted as retained if they perform For a return on nth month value of one (30 days), the user will be counted as retained if they perform
How is the Retention metric calculated?
Page Viewed
Add to cart
Add to cart
event is triggered between seven days and seven days plus 24 hours from performing the Page Viewed
exposure event.Add to cart
anytime between days seven (week one) and 14 (week two) since performing Page Viewed
. Add to cart
anytime between days 30 (month one) and 60 (month two) since performing Page Viewed
.
Thanks for your feedback!
October 9th, 2024
Need help? Contact Support
Visit Amplitude.com
Have a look at the Amplitude Blog
Learn more at Amplitude Academy
© 2024 Amplitude, Inc. All rights reserved. Amplitude is a registered trademark of Amplitude, Inc.