---
title: Databricks
description: "Export Amplitude events, merged Amplitude IDs, custom event definitions, and event transformation definitions to Unity Catalog Delta tables in Databricks."
product: data
lang: en
last_updated: 2026-11-18
token_estimate: 4750
---
# Databricks

Export Amplitude events, merged Amplitude IDs, and data configuration to Unity Catalog Delta tables in your Databricks workspace. The connection authenticates with OAuth Workload Identity Federation (WIF), so Amplitude never holds a long-lived client secret for your workspace.

> For AI agents: a documentation index is available at [/docs/llms.txt](/docs/llms.txt). Append `.md` to any page URL for markdown, or send `Accept: text/markdown`.

## Considerations

- Amplitude delivers exports on a best-effort basis. How soon data lands in your Delta table depends on the run interval you choose. With a 5- or 10-minute run interval, data is typically queryable within 30 to 45 minutes of Amplitude receiving it. Longer intervals add up to one interval, so the default hourly interval can take up to about 1 hour 45 minutes. For data that's typically fresh within minutes, use [express forwarding](#use-express-forwarding).
- You can create multiple Databricks exports from the same Amplitude project. Each export has its own destination table. Event exports also support separate event filters.
- Event filters apply only to event exports. Merged Amplitude ID and data configuration exports don't use the event filter.

### Supported export types

Select one or more export types during setup. You can export merged Amplitude IDs or data configuration without exporting events.

| Export type | Data | Schedule |
| --- | --- | --- |
| Events | Raw events that match your event filter, if you configure one. | Standard: configurable, every hour by default. Express: every few minutes. |
| Merged Amplitude IDs | Records that map merged Amplitude IDs to the IDs they merge into. | Every hour. |
| Event Transformations | Snapshots of your project's event transformation definitions. | Configurable data configuration frequency; every 10 minutes by default. |
| Custom Events | Snapshots of your project's custom event definitions. | Configurable data configuration frequency; every 10 minutes by default. |

## Prerequisites

You need admin or manager privileges in Amplitude, and a role in Databricks that can manage service principals, create Unity Catalog grants, and create an account-level Federation Policy.

Set up the following in Databricks before you open the wizard:

- A Databricks workspace with a Unity Catalog catalog and schema. Capture the workspace host (for example, `https://dbc-xxxx.cloud.databricks.com`).
- A SQL warehouse. Capture the warehouse ID from the Databricks UI under _SQL Warehouses > your warehouse > View JSON_ or the URL.
- A service principal in _Settings > Identity and access > Service principals_. Don't generate an OAuth secret. Capture both:

  - The `application_id` (a UUID). The wizard calls this the `clientId`, and it's the credential Amplitude uses at run time.
  - The numeric service principal ID. The wizard uses this to fill in the Federation Policy CLI command.

> **Tip:** Why Workload Identity Federation?
>
> WIF removes the shared client secret that OAuth machine-to-machine (M2M) authentication requires. It's the more secure default for this integration:
>
> - **No shared secret**: Amplitude never receives a credential that authenticates as your service principal outside of a live export run.
> - **Nothing for you to rotate**: The credential is a short-lived JWT that Amplitude's infrastructure mints and refreshes automatically. There's no password or key on your side that drifts, expires, or leaks through a backup.
> - **One-line revocation**: Deleting the Federation Policy on your service principal ends Amplitude's ability to authenticate immediately, without disturbing other integrations on the same workspace.

The wizard generates the Federation Policy command and the Unity Catalog grants SQL for you. You run them in Databricks during setup.

## Set up the integration

The setup wizard has three steps: _Get started_, _Set up_, and _Verify_. The wizard generates the Databricks CLI command and grant SQL as you type, so you can run them in Databricks without leaving the page.

1. In Amplitude Data, click **Catalog** and select the **Destinations** tab.
2. In the Warehouse Destinations section, click **Databricks**.
3. On the _Get started_ step, select the data to export:

   - **Forward events ingested today and moving forward**: Under **Forwarding mode**, select **Standard** and set the run interval (the default is every hour), or select **Express**, where Amplitude sets the run interval. To forward only events that meet certain criteria, add a filter.
   - **Export all merged Amplitude IDs**: Export the user-merge records.
   - **Data Configuration**: Select **Event Transformations**, **Custom Events**, or both. Set the **Data configuration frequency**. The default is every 10 minutes.

   Click **Next**.
4. On the _Set up_ step, enter a **Connection name** and your **Credential (OAuth WIF)** details:

   - **Workspace host**: the full workspace URL including the scheme, for example `https://dbc-xxxx.cloud.databricks.com`.
   - **Service Principal ID (numeric)**: the numeric ID from the Databricks UI under _Settings > Identity and access > Service principals > your service principal_. The wizard uses this only to fill in the Federation Policy command. Amplitude doesn't store it.
   - **Service Principal application\_id (clientId)**: the UUID from the same Databricks page. This is the credential Amplitude uses at run time.

   Copy the **Federation Policy command** on the right and run it with the Databricks CLI authenticated against your Databricks account, not a workspace. The command tells Databricks to accept JWTs from Amplitude's environment as proof of identity for the service principal. The wizard sets the OIDC issuer in the command automatically for the Amplitude environment your project runs in.
5. In the same step, enter your **Unity Catalog target**:

   - **Catalog** and **Schema**: The Unity Catalog location for your export tables.
   - Table names: Enter a distinct table name for each selected export type. The fields are **Events table name**, **Merged IDs table name**, **Event transformations table name**, and **Custom events table name**. Table names must be unique within this setup, regardless of capitalization. Amplitude creates each table when it first exports data to that table.
   - **SQL warehouse ID**: the warehouse Amplitude uses to run the `COPY INTO` statement.

   Copy the **Unity Catalog grants SQL** on the right and run it in a Databricks SQL editor. Then in the Databricks UI, open _SQL Warehouses > your warehouse > Permissions_ and grant the service principal **CAN USE**. The warehouse permission isn't grantable through SQL.
6. Click **Next** to open the _Verify_ step. Amplitude verifies access and creates a separate export connection for each selected data type. Wait for **Exports created**, then click **Finish**.

Each connection runs on its own schedule and writes to its own table. Exports start on their next scheduled run.

### Revoke Amplitude's access

To stop Amplitude from authenticating as the service principal, delete the Federation Policy you created in step 4:

```bash
databricks account service-principal-federation-policy list <sp-application-id> \
  --profile <your-databricks-account-profile>
databricks account service-principal-federation-policy delete <sp-application-id> <policy-id> \
  --profile <your-databricks-account-profile>
```

Use the same Databricks account profile you used in the **Federation Policy command** during setup. A workspace-scoped profile returns a 401 for these subcommands.

After you delete the policy, the next scheduled run fails with `PERMISSION_DENIED` and Databricks stops issuing workspace tokens for the service principal. To also block data access for any token that's still valid, remove the Unity Catalog grants from step 5.

## Use express forwarding

Express forwarding is in early access.

[Express forwarding](https://amplitude.com/docs/data/express-forwarding) loads raw events into your Delta table every few minutes, so data is typically fresh within minutes. It's available to organizations on the Enterprise plan. With express forwarding, events that Amplitude imports from a data warehouse or cloud storage still arrive hourly.

To use express forwarding, select **Express** under **Forwarding mode** in step 3 of the setup wizard.

Amplitude runs each load as a `COPY INTO` statement on the SQL warehouse you configure. Loads run every few minutes, so with typical auto-stop settings the SQL warehouse doesn't stop between loads. Budget for a SQL warehouse that runs continuously.

Use a dedicated, small SQL warehouse for express forwarding. A serverless SQL warehouse is the best fit because it starts in seconds and needs no cluster management.

If you already forward events with standard forwarding, you can keep that destination running while you set up express forwarding. To run both destinations at the same time:

- Give the express destination its own table. Two destinations that write to the same table load every event twice.
- Give the express destination its own SQL warehouse. Standard forwarding runs larger, less frequent loads, and express forwarding runs small, continuous ones.

After you confirm that data arrives in the express destination's table, delete the standard destination if you no longer need it.

## Databricks export format

### Data location

Amplitude writes each export type to the Unity Catalog table you name during setup. The full identifier is `{catalog}.{schema}.{table_name}`, using the values you enter.

> **Tip:** Route different event slices to different tables
>
> Amplitude's Databricks destination lets you create multiple exports from the same project. Each destination can have its own event filter and its own destination table, so you can fan a project's event stream out into purpose-built tables. For example, set up one destination that filters to checkout events and lands them in `analytics.finance.checkout_events`, and a second that filters to engagement events and lands them in `analytics.product.engagement_events`.

### Event table schema

The event table uses the following Delta columns:

| Column | Type | Description |
| --- | --- | --- |
| `amplitude_attribution_ids` | VARIANT | Hashed attribution IDs on the event. |
| `amplitude_id` | BIGINT | Amplitude ID on the event. Use the Merged User table to resolve user merges. Example: 2234540891. |
| `app` | BIGINT | Project ID found in your project's Settings page. Example: 123456. |
| `city` | STRING | City. Example: "San Francisco". |
| `client_event_time` | TIMESTAMP | Local timestamp (UTC) of when the device logged the event. Example: `2015-08-10T12:00:00.000000`. |
| `client_upload_time` | TIMESTAMP | The local timestamp (UTC) of when the device uploaded the event. Example: `2015-08-10T12:00:00.000000`. |
| `country` | STRING | Country. Example: "United States". |
| `data` | VARIANT | Dictionary where Amplitude stores certain fields such as `first_event` and `merged_amplitude_id`. |
| `device_carrier` | STRING | Device carrier. Example: Verizon. |
| `device_family` | STRING | Device family. Example: Apple iPhone. |
| `device_id` | STRING | The device-specific identifier. Example: C8F9E604-F01A-4BD9-95C6-8E5357DF265D. |
| `device_type` | STRING | Device type. Example: Apple iPhone 5s. |
| `dma` | STRING | Designated marketing area (DMA). Example: San Francisco-Oakland-San Jose, CA. |
| `event_id` | BIGINT | A counter that distinguishes events. Example: 1. |
| `event_properties` | VARIANT | Event properties as semi-structured JSON. |
| `event_time` | TIMESTAMP | Amplitude timestamp (UTC), which is `client_event_time` adjusted by the difference between `server_received_time` and `client_upload_time`: `event_time` = `client_event_time` + (`server_received_time` - `client_upload_time`). Amplitude uses this timestamp to organize events on Amplitude charts. If the difference between `server_received_time` and `client_upload_time` is less than 60 seconds, Amplitude doesn't adjust `event_time` and it equals `client_event_time`. Example: `2015-08-10T12:00:00.000000`. |
| `event_type` | STRING | Event type. |
| `group_properties` | VARIANT | Group properties as semi-structured JSON. |
| `groups` | VARIANT | Group types. For more information, refer to [Account-level reporting](https://amplitude.com/docs/analytics/account-level-reporting). |
| `ip_address` | STRING | IP address. Example: "123.11.111.11". |
| `language` | STRING | Language reported by the device. |
| `library` | STRING | The Amplitude SDK that sent the event. |
| `location_lat` | DOUBLE | Latitude. Example: 12.3456789. |
| `location_lng` | DOUBLE | Longitude. Example: -123.4567890. |
| `os_name` | STRING | OS name. Example: ios. |
| `os_version` | STRING | OS version. |
| `paying` | STRING | True if the user has ever logged any revenue, otherwise none. You can modify the property value with the Identify API. Example: true. |
| `platform` | STRING | Platform that sent the event. |
| `processed_time` | TIMESTAMP | Time Amplitude processed the event. |
| `region` | STRING | Region. Example: California. |
| `sample_rate` | BIGINT | Event sampling rate. |
| `server_received_time` | TIMESTAMP | Server time when Amplitude received the event. |
| `server_upload_time` | TIMESTAMP | Amplitude timestamp (UTC) of when Amplitude servers received the event. Example: `2015-08-10T12:00:00.000000`. |
| `session_id` | BIGINT | The session start time in milliseconds since epoch. Example: 1396381378123. |
| `start_version` | STRING | App version when Amplitude first tracked the user. Example: 1.0.0. |
| `user_creation_time` | TIMESTAMP | Timestamp (UTC) of when Amplitude first observed the user on this project. |
| `user_id` | STRING | A readable ID you specify. Use something that doesn't change. Avoid using the user's email address. |
| `user_properties` | VARIANT | User properties as semi-structured JSON. |
| `uuid` | STRING | Per-event unique identifier Amplitude assigns at ingestion. Use this as the join key to deduplicate events across refreshes or backfills. |
| `version_name` | STRING | App version the SDK reports. Example: 1.0.0. |

### Merged User table schema

The Merged User table records how Amplitude resolves user merges. Use the mapping to associate events with a user's merged identity.

| Column | Type | Description |
| --- | --- | --- |
| `amplitude_id` | STRING | The Amplitude ID that merges into another Amplitude ID. |
| `merged_amplitude_id` | STRING | The Amplitude ID that `amplitude_id` merges into. |
| `merge_event_time` | TIMESTAMP | Event timestamp associated with the user merge. |
| `merge_server_time` | TIMESTAMP | Server timestamp associated with the user merge. |

### Data configuration snapshots

Event transformation and custom event exports contain definitions, such as transformation rules and custom event filters. These exports don't apply the definitions to the raw event table.

Each scheduled run appends a full snapshot of the project's current definitions to the corresponding table. Earlier snapshots remain in the table. Use `snapshot_at` and `export_job_id` to identify rows from the same snapshot.

### Event transformations table schema

Amplitude writes event transformation definitions to the table you enter in **Event transformations table name**.

| Column | Type | Description |
| --- | --- | --- |
| `project_id` | STRING | Amplitude project ID. |
| `snapshot_at` | TIMESTAMP | Time Amplitude produced the snapshot. |
| `export_job_id` | STRING | Identifier of the export run that produced the snapshot. |
| `transformation_id` | STRING | Stable transformation identifier. |
| `name` | STRING | Transformation display name. |
| `description` | STRING | Transformation description. |
| `transform_type` | STRING | Transformation type, such as merged event or merged-event derived property. |
| `merged_props` | VARIANT | Mapping from target properties to source properties for merge transformations. |
| `merge_priority` | VARIANT | Priority rules for merge resolution. |
| `merged_events` | VARIANT | Events that merge to create the transformation. |
| `derived_props` | VARIANT | Properties the transformation creates. This field doesn't represent Amplitude's standalone Derived Properties feature. |
| `derived_props_for_merge_events` | VARIANT | Derived-property mapping for merged events. |
| `is_enabled` | BOOLEAN | Whether Amplitude applies the transformation. |
| `created_at` | TIMESTAMP | Time someone created the transformation. |
| `created_by` | STRING | User who created the transformation. |
| `last_updated_at` | TIMESTAMP | Time someone last modified the transformation. |
| `last_updated_by` | STRING | User who last modified the transformation. |

### Custom events table schema

Amplitude writes custom event definitions to the table you enter in **Custom events table name**.

| Column | Type | Description |
| --- | --- | --- |
| `project_id` | STRING | Amplitude project ID. |
| `snapshot_at` | TIMESTAMP | Time Amplitude produced the snapshot. |
| `export_job_id` | STRING | Identifier of the export run that produced the snapshot. |
| `custom_event_id` | STRING | Stable custom event identifier. |
| `name` | STRING | Canonical custom event name. |
| `display_name` | STRING | Display name in Amplitude. |
| `description` | STRING | Custom event description. |
| `definition` | VARIANT | Definition expression, including filters and groups. |
| `category` | STRING | Optional category or group. |
| `classifications` | VARIANT | Tags or classifications as semi-structured JSON. |
| `suggestion_status` | STRING | Suggestion state, such as suggested, accepted, or rejected. |
| `has_properties` | BOOLEAN | Whether the custom event has custom properties. |
| `is_official` | BOOLEAN | Whether the custom event is an org-official custom event. |
| `is_restricted` | BOOLEAN | Whether the custom event limits access to certain users or roles. |
| `is_autotrack` | BOOLEAN | Whether autotagging generated the custom event. |
| `is_hidden` | BOOLEAN | Whether Amplitude hides the custom event from standard lists. |
| `created_at` | TIMESTAMP | Time someone created the custom event. |
| `created_by` | STRING | User who created the custom event. |
| `last_updated_at` | TIMESTAMP | Time someone last modified the custom event. |
| `last_updated_by` | STRING | User who last modified the custom event. |

## Backfills

For event and merged Amplitude ID exports, the **Backfills** tab on your connection detail page replays a historical date range into the same Delta table as the recurring export. Event backfills use `server_upload_time`; merged Amplitude ID backfills use `merge_server_time`. You can also run manual backfills for express destinations.

Use a backfill to load historical data or recover from data loss in Databricks. Data configuration exports contain current definitions and don't support historical backfills.

