Create a new experiment

The decisions you make in the design phase define your experiment's success. Define your experiment's purpose and goals before you start to gain useful, actionable insights.

For example, you want to run a hypothesis testing experiment with a direction setting of "increase" and a minimum goal (MDE) of 2%. This setting means you expect the metric to increase by at least 2%. If you change the experiment type to Do No Harm, you expect the metric to "not increase by 2%". A good use case for a Do No Harm experiment is launching a service agreement in your app and then testing for a lack of change in user retention.

To create a new experiment, install an SDK or call the evaluation API.

Before you launch, estimate run time and viability with the Duration Estimator.

Create a feature experiment

1. Go to Create > Experiment, and select Feature.
2. Complete the following fields:
   • Name: enter the name of the experiment for future reference.
   • Project: select the project in which this experiment operates.
   • Experiment type: select from the following:
     • A/B test: test one or more variants with a goal of improving a metric. Run A/B tests using hypothesis testing or do-no-harm methodologies. For more information, refer to Define your experiment's goals.
     • Multi-armed bandit: Amplitude allocates an increasing amount of traffic to the winning variant, based on the primary metric, until it reaches 100% allocation.
   • For web experiments, enter the Targeted Page URL where this experiment runs.
3. Optionally, complete the following fields:
   • Key: keys are unique to experiments and identify which experiments a user participates in. You can edit keys until you run the experiment.
   • Evaluation mode: select whether the experiment runs locally or on Amplitude's Experiment servers. For more information, refer to Local evaluation and Remote evaluation.
   • Bucketing unit: select the unit Amplitude uses to assign variants, either User or Group.
4. Select Create.

After Amplitude creates the experiment, configure additional aspects of the experiment. Go to Define your experiment's goals for next steps.