Add variants to your experiment

Your experiment needs at least one variant. Experiment compares a variant experience with the control experience, which is usually your product's current user experience (UX). Experiment measures the performance of the variant against a known quantity: the performance of your app today.

Add and manage variants

Experiment creates your initial variant automatically. By default, this variant uses the name treatment. You can rename the variant to describe the experiment you're running.

To add more variants

1. In the Experiment section, create a new experiment or open an existing one.
2. Click Add a variant.
3. Enter a name for your variant and include a value. By default, the value is a hyphenated (slug) version of the variant name.

4. Add a description of the variant. Be specific so other people can understand what your experiment does.

5. Add an optional payload. A payload is a JSON object that changes a variant's experience dynamically without requiring more code.

   For example, if you're testing a new splash screen on a marketing webpage, early results might suggest different content could improve performance. Instead of editing your codebase, include the changes in a payload, and Experiment applies them automatically.

   Paste or type your code into the window.

6. Click Apply.

You can add any number of variants to an experiment, but adding too many can make it harder for your experiment to reach statistical significance. Keep your experiments to the minimum number of variants needed.

You can drag and drop your variants in any order. The variant with the control label is always the control variant, regardless of its position in the list.

Distribute traffic to your variants

Unless you specify otherwise, Experiment splits traffic evenly between your variants. To send more traffic to specific variations, customize your variant distribution. In your experiment, go to Targeting > Distribution to change the distribution percentage.

Stratified sampling and experiment bias

To spread traffic differently for each user segment you've included:

• Segment 1: Country = USA || 80% treatment, 20% control.
• Segment 2: Country = Canada || 50% treatment, 50% control.

This approach can introduce bias into your experiment results. In general, use uniform allocation ratios across all user segments in an experiment.

Non-uniform allocation ratios often happen inadvertently when users change their rollouts and variants while an experiment runs.

Amplitude Experiment offers stratified sampling (non-uniform allocation ratios) if you need it. Switch the Allow rollout controls per segment toggle to On. This option appears only if you've selected Targeted Users.

This switch appears only for experiments, not for feature flags. Amplitude disables the switch while your experiment is active.

Click Continue to move to the rollout phase.

Rollout percentage

The next step sets the rollout percentage for this experiment. The rollout percentage is the percentage of users in the experiment's user segments who take part in the experiment. Find the rollout percentage in the Rollout section of the experiment design panel.

Enter the percentage of your audience eligible for bucketing into the experiment. If you roll your experiment out to less than 100% of your users, the remaining users see your default product experience and aren't included in any experiment calculations.

Experiment evaluates users in rule-based user segments before users not covered by a user segment. Experiment evaluates individual user or device IDs before both.

Next, finalize your experiment's statistical settings.