Confluent (Kafka REST)

Amplitude Data's Confluent integration streams your Amplitude event data to a Kafka topic using the Confluent Kafka REST v3 produce API. Use it to land Amplitude events in Confluent Cloud or a self-managed Confluent REST Proxy without running a connector of your own.

Considerations

• This destination speaks Confluent's Kafka REST v3 API. It works with Confluent Cloud and with a self-managed Confluent REST Proxy or Confluent Server. It doesn't work with Redpanda, which serves only the older v2 routes, or with Amazon MSK and open-source Apache Kafka, which expose no HTTP produce API at all.
• The REST endpoint must reach the public internet and resolve to a public IP address. Amplitude rejects localhost, internal-only hostnames, and private address ranges, so you can't point this destination at a VPC-internal proxy or a locally hosted REST Proxy.
• Amplitude puts the same JSON that the Amplitude Export API documents in the record value.
• Confluent is a near real-time streaming destination, so it counts toward your Event Streaming Usage.
• This destination supports Identify Forwarding. Anytime you make an Identify call to Amplitude, Amplitude forwards that user information. Refer to the Identify documentation for more information.
• Amplitude produces one record per request. Confluent documents the non-streaming produce mode at fewer than 100 requests per second per connection, so plan the volume you route to this destination accordingly.
• Amplitude encrypts your API key and secret at rest, then sends them as HTTP basic auth credentials on every produce request.

Set up the integration

Confluent setup

Complete these steps in Confluent Cloud (or on your self-managed cluster) before you create the destination in Amplitude.

1. Create the topic

Create the topic that receives Amplitude events, or choose an existing one. Amplitude doesn't create topics, so the topic must exist before Amplitude sends the first event, unless your cluster has topic auto-creation enabled.

Choose the partition count based on your expected volume. Amplitude sets a record key on every event, so events for the same user consistently land on the same partition.

2. Create a service account and API key

1. Create a service account for Amplitude.
2. Create a Kafka API key and secret for that service account, scoped to the cluster that hosts your topic.
3. Store the secret somewhere safe. Confluent shows it only once.

3. Grant write access to the topic

Give the service account permission to produce to the topic. With the Confluent CLI:

confluent kafka acl create \
  --allow \
  --service-account sa-xxxxxx \
  --operations write \
  --topic your-topic-name \
  --cluster lkc-xxxxxx \
  --environment env-xxxxxx

Amplitude only produces, so write is the only operation the destination needs.

4. Collect the connection details

Gather these values, which you enter in Amplitude:

Include the API version in the endpoint. Confluent Cloud and Confluent Server serve /kafka/v3, while a standalone REST Proxy serves /v3. Amplitude appends only the cluster and topic path segments.

Amplitude setup

1. In Amplitude Data, click Catalog and select the Destinations tab.
2. In the Event Streaming section, click Confluent (Kafka REST).
3. Enter a sync name, then click Create Sync.
4. Click Edit, then enter the following:
   • Kafka REST endpoint: the endpoint from Confluent, including the /v3 or /kafka/v3 suffix.
   • Kafka cluster ID: for example, lkc-xxxxxx.
   • Topic: the topic that receives the events.
   • API key and API secret: the credential you created for the service account. Leave both blank only if your REST Proxy runs without authentication.
   • Event field to use as the record key: optional. Leave it blank to use user_id.
5. Use the Send events filter to select the events you want to send. You can send all events, but Amplitude recommends choosing the most important ones.
6. When finished, enable the destination and save.

Record format

Amplitude sends one produce request per event to {endpoint}/clusters/{clusterId}/topics/{topic}/records, with a body like this:

{
  "key": {"type": "STRING", "data": "user-123"},
  "value": {"type": "JSON", "data": {"user_id": "user-123", "event_type": "Checkout Completed", "...": "..."}}
}

The value is the full Amplitude event as a JSON object, matching the Export API schema.

The key controls partitioning. Amplitude uses the field named in Event field to use as the record key, or user_id when that setting is blank. If the chosen field is missing or empty on an event, Amplitude falls back to device_id, which keeps anonymous traffic on a stable partition. If neither field has a value, Amplitude omits the key and Confluent assigns the partition.

Troubleshooting