Test and launch your experiment
Before any users view your experiment, confirm that the variants look and function as intended. Because Experiment lets you assign specific variants by user ID, device ID, or cohort, you can confirm that Amplitude serves test devices the correct variants when they enter your experiment.
To exclude users from an experiment or flag, add them to the OFF variant.
On the Overview page for your experiment, review the Overview, Delivery, Variants, and Targeting sections. Confirm each section matches your plan.
Click Test Instrumentation to send the experiment's variants to the testers you designated when you configured the experiment's audience.
Test Instrumentation and targeting
When you test your instrumentation, Amplitude ignores the target segments you configured in the experiment. Test instrumentation sends variants only to the Testers.
Launch your experiment
When you're satisfied that your experiment works as intended, click Start Experiment.
You can set an end date for the experiment or accept the default Experiment analysis range.
Clicking Start Experiment is the only way to activate your experiment. If you change the start date for the experiment, the experiment doesn't activate automatically on the new date.
After your experiment runs, you can make a decision on your experiment when it reaches statistical significance or its end date.
Schedule your experiment
To schedule the experiment for a later launch, expand the Start Experiment menu and click Schedule start. In the modal that appears, set the date and time to begin the experiment.
Experiment start and variant delivery
When a scheduled experiment reaches its start time, the experiment may take up to one hour to begin exposing users to variants.
QA after rollout
After rollout, you can track how many users Amplitude exposed to each variant on a daily basis.
Go to Experiments > your experiment > Activity tab > Diagnostics to view how many users Amplitude exposed to each variant.
This view is a useful way to QA the assignment process. If one variant enrolls significantly more or significantly fewer users than expected, the difference may indicate an issue to investigate.
If you spot outliers or anomalies that concern you, click into the chart or information to investigate the potential causes. To learn more about understanding anomalies, refer to this article on Root Cause Analysis.
For a deeper validation of your experiment's instrumentation and assignment logic, run an A/A test before launching a full A/B test.
Was this helpful?