Gyrfalcon - BigQuery Import Migration to Temporal
BigQuery Import has been fully migrated from Falcon's custom orchestration to Temporal workflows via Gyrfalcon - a new architecture delivering faster ingestion, lower costs, better observability, and an improved customer experience.
What it is
Description: BigQuery Import has been fully migrated from Falcon's custom orchestration (Falcon+Vacuum) to Temporal workflows via Gyrfalcon - a new architecture that replaces our legacy scheduling, state management, and worker coordination with a managed orchestration platform. This eliminates the most complex and maintenance-heavy parts of Falcon's import pipeline, delivering faster ingestion, lower costs, better observability, and an improved customer experience.
Why: Falcon's import pipeline relied on custom-built orchestration across multiple services (Falcon scheduler → batch jobs → worker jobs → Vacuum), creating significant operational overhead and latency. Customers experienced slow ingestion due to suboptimal poll-based scheduling, had no visibility into empty jobs, and the team spent disproportionate time debugging opaque pipeline failures. Gyrfalcon consolidates this into a single orchestration layer with Temporal, delivering measurable improvements: • Ingestion latency reduced by up to 60% - large customers saw ingestion time reduced by 1 hour for a single import job; smaller customers reduced from ~15-20 min to ~5-7 min • Projected ~$10K/month infrastructure savings • $1.5K/month ingestion cost reduction already realized, with additional savings to come • Proactive anomaly detection with regression-based monitoring • Improved debugging through full workflow history visible in Temporal UI • Unified metering experience replacing fragmented per-connector metrics • Faster development velocity with standardized Temporal workflow architecture • Reduced maintenance and opportunity costs by consolidating orchestration • Zero-downtime reversible migration with instant rollback capability
Where: All existing and new BigQuery Import sources (except for Amplitude projects with event streaming destinations). Customers see an improved import details UI with job-level status breakdowns, visibility into empty jobs, and near-instant completion times for small imports.
When: GA is now available for all BigQuery Import customers (except those with event streaming destinations). Other Data Warehouse and Cloud Storage integrations are planned throughout 2026.
Customers Impacted: All BigQuery Import customers (232+ source connections, up to 7.8B events per day), including top-tier customers processing 100M+ events per job.
Available in EUDC: Yes, fully rolled out in US and EU.
KPIs: Import Volume and Number of Sources metrics are tracked.