Make a decision on your experiment

After your experiment reaches statistical significance or its end date, decide what to do next. This article covers how to end your experiment and the options available.

End your experiment

End the experiment when you reach its end date or when it reaches statistical significance. To end it, click Complete Experiment.

Then choose one of the following:

• Roll out the winning variant.
• Roll back everything and return to a pre-experiment state.
• Continue the experiment.

You can revisit this decision later.

What happens when your experiment ends

When you complete your experiment, Amplitude stops collecting new data for analysis. The experiment stays in your experiment list but no longer assigns new users to variants. Historical data and analysis stay available for review.

If you roll out or roll back, Amplitude applies the changes you select. If you continue the experiment, it stays active until the new end date you specify.

Roll out variants

When you roll out the winning variant to all users, Amplitude:

• Sets the rollout percentage to 100%.
• Changes the distribution weight to 100 for the winning variant and 0 for all other variants.
• Disables sticky bucketing (sets it to false).

If you roll out to a custom percentage of users, Amplitude doesn't apply these automatic changes. You must apply changes manually after confirming your rollout decision.

You can also roll out to "only the targeted users." This option helps avoid overgeneralizing your experiment's results. For example, you target users in the United States and find a 5% improvement. That result doesn't mean a rollout to all users produces a 5% lift outside the United States. You might experience a 5% lift for U.S. users but a -2% lift for all other users.

Roll back an experiment

When you roll back your experiment, Amplitude:

• Turns the flag off.
• Sets percentage rollouts to 0%.

Continue your experiment

To continue running your experiment, enter a new end date to gather more data and click Start Experiment.

To restart your experiment with fresh results (for example, after instrumentation issues affected data quality), create a new experiment run. A new run excludes previous user data from monitoring and analysis.

Clean up feature flags

After you decide on your experiment, clean up your feature flags to avoid unnecessary overhead and confusion.

Deactivate or archive your experiment

After you roll out, roll back, or continue your experiment, deactivate or archive it in Amplitude. Archiving removes unnecessary logic and prevents accidental reactivation or analysis confusion.

To deactivate or archive your experiment:

1. Navigate to your experiment in Amplitude Experiment.
2. Click the menu next to Complete Experiment or Turn off flag.
3. Select Archive to archive the experiment.

Archived experiments stay in your Experiment List with an archived label. You can unarchive them when needed.

Remove experiment code

When you roll out the winning variant, work with your engineering team to implement the winning experience directly in your production code base. Implementing the variant in code removes the need to keep the experiment active with 100% traffic allocation.

After your engineering team implements the chosen variant in code, remove the experiment logic from your codebase. Removing the logic reduces technical debt and improves performance.