From Job Clusters to DBSQL Serverless: How GetYourGuide Improved DBT Pipelines

At GetYourGuide, we used to run ETL transformations with an internal tool called Rivulus on top of Databricks job clusters. When we standardized on dbt-databricks, we migrated these workloads to DBSQL Serverless and had to rethink how we design and operate our pipelines. We'll walk you through what changed and what we wished to know before we started.What changed for us:

• Pipelines run ~60% faster and costs dropped ~20%
• Time to production for new models went from days to minutes, backfills from weeks to hours
• ~4 months of annual overhead from DBR version upgrades disappeared
• 100% of our core models are now automatically documented with AI
• Finance and marketplace teams now rely on fresher data with less friction

Attendees will leave with:

• A clear view of how the migration to DBSQL Serverless was performed
• How we turned curve balls into wins along the way
• Concrete steps to apply similar ideas in their own dbt-databricks setups, including using ai_query for automated model documentation