Test and launch your experiment
Before any users view your experiment, confirm that the variants look and function as intended. Because Experiment lets you assign specific variants by user ID, device ID, or cohort, you can confirm that Amplitude serves test devices the correct variants when they enter your experiment.
To exclude users from an experiment or flag, add them to the OFF variant.
On the Overview page for your experiment, review the Overview, Delivery, Variants, and Targeting sections. Confirm each section matches your plan.
Click Test Instrumentation to send the experiment's variants to the testers you designated when you configured the experiment's audience.Test Instrumentation and targeting
When you test your instrumentation, Amplitude ignores the target segments you configured in the experiment. Test instrumentation sends variants only to the Testers.
Launch your experiment
When you're satisfied that your experiment works as intended, click Start Experiment.
You can set an end date for the experiment or accept the default Experiment analysis range.
Clicking Start Experiment is the only way to activate your experiment. If you change the start date for the experiment, the experiment doesn't activate automatically on the new date.
Schedule your experiment
To schedule the experiment for a later launch, expand the Start Experiment menu and click Schedule start. In the modal that appears, set the date and time to begin the experiment.
Experiment start and variant delivery
Amplitude exposes users to variants immediately when they arrive at or after the scheduled start time. Users who requested a variant in the hour before the start time may continue to see the pre-start response until their cached response expires, up to one hour later. Unlike a manual flag or experiment update, a scheduled start doesn't trigger CDN cache invalidation, so the existing 60-minute cache TTL applies to those earlier requests. For details, refer to Cache invalidation.For example, for an experiment that starts at 2:00 PM, a user who visited at 1:30 PM continues to see the pre-start response until 2:30 PM, when their cached response expires. A user who arrives at 2:00 PM sees a variant right away.
QA after rollout
After rollout, you can track how many users Amplitude exposed to each variant on a daily basis.
Go to Experiments > your experiment > Activity tab > Diagnostics to view how many users Amplitude exposed to each variant.
This view is a useful way to QA the assignment process. If one variant enrolls significantly more or significantly fewer users than expected, the difference may indicate an issue to investigate.
If you spot outliers or anomalies that concern you, click into the chart or information to investigate the potential causes. To learn more about understanding anomalies, refer to this article on Root Cause Analysis.For a deeper validation of your experiment's instrumentation and assignment logic, run an A/A test before launching a full A/B test.Was this helpful?