Express forwarding
이 페이지는 아직 귀하의 언어로 번역되지 않았습니다. 현재 작업 중이므로 곧 다시 확인해 주십시오.
이 기능은 Early Access 버전입니다. 이 기간 동안에는 기능의 일부가 계속 개발될 수 있으며, 이 문서는 항상 최신 상태가 아닐 수 있습니다. 질문이 있으시면 다음으로 문의하십시오: Amplitude Support.
Express forwarding sends raw events to Snowflake or Databricks every few minutes. Data is typically queryable in your table within minutes of Amplitude receiving it, compared with 30 to 45 minutes for standard forwarding. Express forwarding is in early access and available to organizations on the Enterprise plan.
Choose a forwarding mode
Standard forwarding loads a time range only after Amplitude finishes processing every event in it. Your table fills in server_upload_time order, but each run waits for the slowest part of that processing. As a result, data is less fresh, and a delay anywhere in processing holds back the whole run.
Express forwarding loads events as soon as they're ready. A delay in one part of processing holds back only the events it affects. As a result, data is much fresher, both in typical conditions and when part of processing is slow. In rare cases, an event arrives after events with a later server_upload_time.
Express forwarding fits most use cases. Choose standard forwarding when your downstream jobs need events in strict server_upload_time order and can accept the delay.
| Standard forwarding | Express forwarding | |
|---|---|---|
| Run interval | You choose, from every 5 minutes to once a day | Every few minutes, set by Amplitude |
| Typical freshness | 30 to 45 minutes with a 5- or 10-minute run interval. Longer intervals add up to one interval. | Within minutes |
| When processing is delayed | The whole run waits. | Only the delayed events wait. |
| Event order | Strict server_upload_time order | Usually in order, with rare exceptions |
| Events from warehouse and cloud storage imports | Same run interval as other events | Hourly |
| Destinations | All warehouse and cloud storage destinations | Snowflake and Databricks |
| Availability | Generally available | Early access, Enterprise plan |
To choose a mode, answer these questions in order and stop at the first yes:
- Is your organization on a plan other than Enterprise? Use standard forwarding.
- Is the destination something other than Snowflake Direct Load or Databricks? Use standard forwarding.
- Does every event in the project come from warehouse or cloud storage imports? Use standard forwarding. Express forwarding can't make imported data fresher.
- Do your downstream jobs need events in strict
server_upload_timeorder? Use standard forwarding. - If you answered no to every question, use express forwarding.
Run interval and data freshness
Freshness is the time from when Amplitude receives an event until you can query it in your table. The run interval is how often Amplitude loads data. The two aren't the same: each run loads the events that are ready, so freshness depends on the run interval plus the time Amplitude and your warehouse need to process and load the data.
With express forwarding, Amplitude sets the run interval and runs every few minutes, so data is typically fresh within minutes. Under heavy load, some events take longer.
Your warehouse compute settings, your warehouse's cloud region, your cloud provider, and your Amplitude data region also affect freshness.
Which events arrive faster
Express forwarding speeds up delivery of every event that Amplitude receives directly, including events from:
- Amplitude SDKs
- The HTTP V2 API
- The Batch Event Upload API
- Partner integrations that send events to Amplitude
Events that Amplitude imports from a data warehouse or cloud storage arrive in the same table on an hourly schedule. Express forwarding doesn't change when Amplitude imports these events.
During setup, Amplitude checks your project's sources and shows one of three results:
- If every source is fast, express forwarding applies to every event in the project.
- If the project also imports events from a data warehouse or cloud storage, express forwarding speeds up the other events, and imported events still arrive hourly. Select Review sources to see which events these are.
- If every event comes from imports, express forwarding can't make this data fresher, so you can't select Express.
If Amplitude can't determine your project's sources, Express stays available with a note to review your sources.
How data from express forwarding differs
Events can arrive out of order
Express forwarding doesn't wait for all of Amplitude's processing to catch up. In rare cases, a run loads an event with an earlier server_upload_time than events already in your table.
If an incremental job reads only rows newer than the latest server_upload_time it has seen, the job can miss these late events. Re-read a short lookback window on each run, or treat a day as complete only after its daily integrity check.
Your table can contain duplicate rows
Neither forwarding mode guarantees that Amplitude loads each event exactly once. In rare cases, the same event appears more than once in your table, with express or standard forwarding. The daily integrity check removes these duplicates, typically within a day.
Amplitude checks each day after it ends
Amplitude runs a daily integrity check on each express destination. A day runs from midnight to midnight UTC, by server_upload_time. After a day ends, Amplitude compares the events it received for that day with the rows in your table. If rows are missing or duplicated, Amplitude replaces that day with an automatic backfill.
During the day, Amplitude doesn't guarantee that any timestamp in your table has all of its events. Use the daily integrity check to confirm a day is complete.
Backfills
When you create an express destination, Amplitude automatically loads the 3 days before the day you create it. To load older data, run a manual backfill from the destination's Backfills tab. You can run manual backfills for Snowflake and Databricks destinations.
A backfill replaces whole UTC days in your table, so it doesn't duplicate rows that are already there.
Express forwarding starts loading events when you create the destination, so the automatic load doesn't cover the part of the creation day before that moment. The daily integrity check for the creation day fills in those events. Treat the creation day as complete only after its integrity check runs.
Limits
- Forwarding mode is fixed at creation. You choose Standard or Express when you create a destination. Afterward, you can't switch modes or convert an existing standard destination to express forwarding.
- Running standard and express together depends on the destination.
- Snowflake: a project can have only one Snowflake Direct Load raw-event destination. To move to express forwarding, delete the standard destination, then create an express destination. The new destination loads the previous 3 days automatically. Use a manual backfill for older data.
- Databricks: a project can have a standard and an express destination at the same time. Each destination must write to its own table. Use a separate SQL warehouse for each destination, because the two modes load data differently.
- Raw events only: express forwarding sends the event table. It doesn't send merged Amplitude IDs or data configuration snapshots.
- Warehouse compute: express forwarding loads data into your warehouse every few minutes. Warehouse compute is typically similar to standard forwarding with a 5-minute run interval. Compute can be higher than with longer standard intervals, at low data volumes, or if your warehouse suspends between loads.
- Pauses longer than 3 days: after a pause or repeated failures, an express destination catches up on its own if it resumes within 24 hours. If it resumes within 3 days, the daily integrity checks fill in the missed days automatically, so treat those days as complete only after their integrity checks run. A destination can stay paused or failing for longer, for example after its credentials expire. In that case, run a manual backfill for the UTC days more than 3 days before the destination resumes.
Get access
Express forwarding is in early access and available to organizations on the Enterprise plan. When you create a Snowflake (Direct Load) or Databricks destination, Express appears under Forwarding mode, labeled Beta. Organizations on other plans see the standard setup, with no Express option. To request access for one of these organizations, contact your Amplitude account team.
Common questions
Should I use express or standard forwarding?
Use express forwarding unless your downstream jobs need events in strict server_upload_time order. Go to Choose a forwarding mode.
Why isn't my data fresher after I moved to express forwarding?
Express forwarding doesn't speed up events that Amplitude imports from a data warehouse or cloud storage. These imported events still arrive hourly. Select Review sources in the setup flow to check your project's sources.
Can I switch an existing destination to express forwarding?
No. Forwarding mode is fixed when you create a destination. Instead, create a new destination and select Express under Forwarding mode:
- Snowflake: delete the standard destination first. A project can have only one Snowflake Direct Load raw-event destination. The new destination loads the previous 3 days automatically.
- Databricks: keep the standard destination running while you set up and verify the express destination. Use a separate table and SQL warehouse for the express destination, then delete the standard destination.
How do I know when a day's data is complete?
Wait for the daily integrity check for that day. If it finds missing or duplicate rows, Amplitude replaces the day with an automatic backfill. Until then, treat the day's data as incomplete.
What happens if my destination is paused or failing?
If the destination resumes within 24 hours, express forwarding catches up on its own and loads every event it missed. If it resumes within 3 days, the daily integrity checks backfill the days it missed, so treat those days as complete only after their integrity checks run. After a longer pause, run a manual backfill for the UTC days more than 3 days before the destination resumes.
이 내용이 도움이 되었나요?
For AI agents: a documentation index is available at /docs/llms.txt. Append .md to any page URL for markdown, or send Accept: text/markdown.