Cohort Targeting | Amplitude Experiment

A cohort is a static or dynamic set of users or groups defined in Amplitude. For experiment use cases, cohorts are particularly useful for advanced audience targeting. That said, cohorts aren't always the best solution for targeting, so understanding how cohort targeting works with local or remote evaluation is important.

Remote evaluation

When you target a cohort in a remote evaluation flag, the cohort is automatically synced to the Amplitude Experiment destination. For dynamic cohorts, this sync runs hourly by default. This means that dynamic cohorts targeted in remote evaluation aren't real-time. For example, if you target a cohort of users or groups who performed a Sign Up event, users are targeted within an hour of performing the event.

Cohorts targeted for remote evaluation may have a propagation delay on the initial sync or large change, depending on the size of the difference. For example, the first sync of 10-million-user cohort is likely to take a lot more time than later syncs.

Use remote evaluation cohort targeting if:

You are targeting users based on user behavior or properties that aren't available in Experiment targeting segments.
You are ok with some targeting delay introduced by cohort sync intervals.
You want to target groups.

Don't use remote evaluation cohort targeting if:

Users must be targeted in real-time.

Local evaluation

Local evaluation flags and experiment that are deployed to up-to-date server-side SDKs can also target cohorts. When you target a cohort in a local evaluation flag, the cohort is automatically synced to the Experiment Local Evaluation destination. For dynamic cohorts, this sync runs hourly by default. This means that dynamic cohorts targeted in local evaluation aren't real-time. For example, if you target a cohort of users who performed a Sign Up event, users will be targeted within an hour of performing the event--not immediately after.

Cohorts only support User IDs

Local evaluation cohorts only sync user IDs to the SDKs. This means that to target cohorts in local evaluation flags, you must include a user ID in the user object passed to the evaluate function.

SDK Support

Server-side SDKs can target cohorts if configured to do so.
On initialization, configure the cohort sync configuration with the project API and secret key to enable local evaluation
cohort downloading and targeting.

SDK	Cohort Targeting	Version
Node.js	✅	`1.10.0+`
Ruby	✅	`1.5.0+`
JVM	✅	`1.4.0+`
Go	✅	`1.7.0+`
Python	✅	`1.4.0+`
PHP	❌	N/A

const experiment = Experiment.initializeLocal('DEPLOYMENT_KEY', {
  // (Recommended) Enable local evaluation cohort targeting.
  cohortSyncConfig: {
    apiKey: 'API_KEY',
    secretKey: 'SECRET_KEY'
  }
});

// (1) Initialize the local evaluation client with a server deployment key.
LocalEvaluationClient experiment = Experiment.initializeLocal("<DEPLOYMENT_KEY>",
    // (Recommended) Enable local evaluation cohort targeting.
    LocalEvaluationConfig.builder()
        .cohortSyncConfig(new CohortSyncConfig("<API_KEY>", "<SECRET_KEY>"))
        .build());

client := local.Initialize("DEPLOYMENT_KEY", &local.Config{
  // (Recommended) Enable local evaluation cohort targeting.
  CohortSyncConfig: &local.CohortSyncConfig {
    ApiKey: "API_KEY",
    SecretKey: "SECRET_KEY"
  }
})

experiment = Experiment.initialize_local("DEPLOYMENT_KEY", LocalEvaluationConfig(
  # (Recommended) Enable local evaluation cohort targeting.
  cohort_sync_config=CohortSyncConfig(api_key="API_KEY", secret_key="SECRET_KEY")
))

experiment = AmplitudeExperiment.initialize_local('DEPLOYMENT_KEY',
  # (Recommended) Enable local evaluation cohort targeting.
  AmplitudeExperiment::LocalEvaluationConfig.new(
    cohort_sync_config: AmplitudeExperiment::CohortSyncConfig.new(
      api_key: 'API_KEY',
      secret_key: 'SECRET_KEY'
    )
  )
)

Troubleshooting

Troubleshooting cohort targeting can challenging due to the asynchronous nature of dynamic cohorts and cohort syncs in general. If you find that your experiment isn't targeting users who should be in the targeted cohort ...

For local evaluation, check that the SDK version supports local evaluation cohort targeting, and that the cohort sync config has been set on initialization.
Check that the cohort has the required sync (Amplitude Experiment for remote evaluation, Experiment Local Evaluation for local evaluation).
Check that the cohort contains the expected user. If the user is contained in the current cohort, check the sync history of the cohort -- is it possible that the user was added to the cohort in a sync after the evaluation occurred?
Check that the user info passed to fetch/evaluate is correct.