Amazon Kinesis (Cohort)

Amplitude Data's AWS Kinesis Cohort integration lets you stream your cohort membership updates straight to a Kinesis stream.

Considerations

• This integration is only available for customers who have paid plans with Amplitude.
• You must enable this integration for each Amplitude project you want to use it in.
• You should have an understanding of Amplitude Audiences and Behavioral Cohorts before setting up this integration.
• Cohort updates can be sent via all the supported cadences (Daily, Hourly, Real Time).
• All the existing Real Time Cohort Sync limits apply here.

Payload

The payload sent to Pub/Sub is a JSON object. Example:

 1{
 2  "cohort_name": "Test Cohort",
 3  "cohort_id": "gs72ns",
 4  "in_cohort": true,
 5  "computed_time": "1685748245",
 6  "message_id": "some-message-id:54",
 7  "users": [{
 8    "user_id": "user1@amplitude.com",
 9    "user_properties": {
10      "city": "San Francisco",
11      "fav_animal": "Bat"
12    }
13  },
14    {
15      "user_id": "user2@amplitude.com",
16      "user_properties": {
17        "city": "Seattle",
18        "fav_animal": "Cat"
19      }
20    }]
21}

• cohort_name: String. the name of the audience
• cohort_id: String. the unique ID of the audience, which can also be found in URL when viewing it on Amplitude
• in_cohort: Boolean. Indicating this batch of users is “entering” or “exiting” an audience. Each message will only have one of either state but not both.
• compute_time: String. The Epoch timestamp when Amplitude re-computes the audience. Due to the nature of Kinesis streaming, it’s impossible to enforce order upon receiving the message. The team can leverage this to resume the order.
  • We kept it as a string rather than a number because depending on the programming language/platform the team is using, a JSON number can be interpreted as a float and lose precision.
• message_id: String. The unique id of each message.
• users: List. The users batch. Each user will be represented as a JSON object
  • user_id: String. the id of the user
  • (optional) user_properties: Map. A list of user properties to include, with key as the property name and value as the property value.

Setup

Prerequisites

Before you get started, create a Kinesis topic in your AWS account, and an IAM role that will be used by Amplitude to publish cohort updates to the stream.

Kinesis setup

1. Create a Kinesis stream

Create a Kinesis stream in the AWS console.

2. Create an IAM role

Create an IAM role that gives Amplitude permission to write directly to your AWS Kinesis Stream.

• For Account ID, specify: 358203115967. This is Amplitude's AWS account used to export events.
• Finalize the role without assigning any policies.
• The Trust Relationships for the new should look like this:

 1{
 2  "Version": "2012-10-17",
 3  "Statement": [
 4    {
 5      "Effect": "Allow",
 6      "Principal": {
 7        "AWS": "arn:aws:iam::358203115967:root"
 8      },
 9      "Action": "sts:AssumeRole"
10    }
11  ]
12}

3. Create an IAM policy

1. Create an IAM Policy to give Amplitude permission to write to your Kinesis Stream.

2. Select the Create Policy from JSON option and use the following template policy in the Policy Document field. Be sure to change the {account-id} to your account-id, and replace {region} and {stream-name} with the applicable values. Also replace {role-name} with the role just created.

    1{
    2    "Version": "2012-10-17",
    3    "Statement": [
    4        {
    5            "Effect": "Allow",
    6            "Action": [
    7                "kinesis:PutRecord",
    8                "kinesis:PutRecords",
    9                "iam:SimulatePrincipalPolicy"
   10            ],
   11            "Resource": [
   12                "arn:aws:kinesis:{region}:{account-id}:stream/{stream-name}",
   13                "arn:aws:iam::{account-id}:role/{role-name}"
   14            ]
   15        }
   16    ]
   17}

3. Assign the new policy to the role.

   1. Open the role
   2. Select the Attach Policies option, and select the new policy.

Make note of the role ARN, stream region, and stream name. You need these to set up the destination in Amplitude.

Amplitude setup

1. In Amplitude Data, click Catalog and select the Destinations tab.
2. In the Cohort section, click on Kinesis Data Stream.
3. Click Add another destination.
4. Enter the Display Name for this destination. This is just a name to identify the destination when syncing the cohort.
5. Enter the stream name the AWS region and the role ARN that was created above.
6. Save the destination.

Syncing Cohort Updates

1. In Amplitude, open the cohort you want to export.
2. Click Sync, and choose Kinesis Data Stream.
3. Select the destination created above.
4. Select the sync frequency you need.