Sticky bucketing

Sticky bucketing checks whether a user already experienced a variant of your experiment. If so, Experiment assigns the current value of the user property (the last variant the user saw) to the user. Sticky bucketing keeps users in their original variants unless you change the parameters of the experiment.

Sticky bucketing often defends against variant jumping, which occurs when a user experiences two or more variants for a single flag or experiment. Enabling sticky bucketing doesn't guarantee that users never experience variant jumping. Variant jumping can occur if your experiment includes both a logged-out and a logged-in experience, because a user can have different logged-in and logged-out Amplitude IDs.

How sticky bucketing works

To turn sticky bucketing on or off, open your experiment, click the pencil icon, and go to Advanced (Optional) > Bucketing Options.

When you enable sticky bucketing, Experiment checks whether a user already has a value for the user property associated with the experiment. If so, Experiment assigns the user the current value of the user property (the last variant the user saw). Otherwise, Experiment re-evaluates the user.

Amplitude doesn't sticky bucket users to the Off variant. For more about evaluation and exposure, refer to the evaluation flow chart and local evaluation targeting capabilities.

When to use sticky bucketing, and when not to

This section provides examples of when to enable sticky bucketing versus when not to. This list isn't exhaustive. In some cases, results stay the same whether sticky bucketing is on or off. An example is an experiment that targets everyone who views your home page, where you don't change any experiment controls while the experiment runs.

When to enable sticky bucketing

• You want to give the user a consistent experience, even if the user property you target changes. For example, if you run an experiment only in the United States, sticky bucketing ensures users keep the same variant if they travel outside the country.
• You want to decrease the percentage rollout of an experiment where the treatment group isn't performing well, but you don't want to move users from treatment or control to the group that never saw either variant (the 'Off' variant). Sticky bucketing keeps users in their assigned groups even after you change the percentage rollout. Increasing or decreasing percentage rollout doesn't cause users to move between treatment and control.
• You want to target users for a specified duration (for example, two weeks) and then stop targeting new users, while maintaining the original assignments for any users Amplitude already bucketed. You might do this to study the long-term effects of a particular treatment on user behavior, or if you aren't sure about the quality of one or more of the experiment's treatments. Enable sticky bucketing at the beginning of the experiment, with a 50/50 split. After the duration passes, change the rollout percentage to zero.
• You want to sunset a failed experiment, but ensure users already bucketed into an experience still get that experience. Enable sticky bucketing and set the rollout percentages to zero.

When not to enable sticky bucketing

• You want the user's experience to change as the targeted user property changes. To continue the previous example, if you run an experiment in the United States, you may not want users to have the same experience when they travel abroad.
• Your experiment drives free users to become paid users and relies on earning rewards. After these users convert, you no longer need to offer a reward. If you enable sticky bucketing here, those users keep the free experience even after upgrading to paid.
• You want to enforce a "cool down" period between giving discounts. To limit discounts to one every seven days per user, add a seven-day filter to the targeting criteria. If a user received a discount within that period, the flag evaluates to off. This filter prevents the user from collecting another discount before you want them to.
• You're rolling out or rolling back a variant. When you enable sticky bucketing and change the traffic allocation, you get a weighted average between the old and new allocation, because users already bucketed stay in their bucket. Your experiment takes some time to reach the allocations you want.

Verify if sticky bucketing was enabled for a specific user

Follow these steps to determine whether sticky bucketing applied to a user:

1. Check the Experiment Assignment events in the user's event stream. This check works only if you haven't blocked the Experiment Assignment events in Data.
2. Find the event property with .details that corresponds to the experiment flag key of interest. This property shows the version of the evaluated flag and which targeting rule applies to the user. The property also helps with debugging assignment issues.

For example, the properties above show that Amplitude assigns the user to off for the lp-app-downloads flag because the device family isn't iOS. The properties also show that this is the 21st version of the flag, the applied rule is non-iOS users, and the user fails the first rule-based targeting filter, so Amplitude applies the second one instead.

In this example, sticky bucketing is on and Amplitude bucketed the user to the 14th version of the signup-ux-updates flag, where the experiment served the phone-number-removed variant. The flag version helps with debugging when the flag changes. Experiment shows the assignment event for all active flags in that project, but shows the exposure event on a per-flag basis. If you don't find an event property that corresponds to the flag you want, check the [Experiment] Environment Name field and confirm it matches the deployment your flag belongs to.