Session

Lakeflow Connect: Cutting Enterprise Ingestion Costs by up to 90% on Databricks

Overview

ExperienceIn Person
TrackData Engineering & Streaming
IndustryManufacturing, Consulting & Services, Retail & Consumer Goods
TechnologiesDatabricks SQL, Lakeflow, Unity Catalog
Skill LevelIntermediate

Enterprise CRM-to-lakehouse pipelines built on traditional third-party ingestion tools can be expensive, time-consuming to implement, and often deliver stale data batches. We replaced this with Databricks Lakeflow Connect for a Fortune 500 client — significantly reducing monthly costs, cutting setup time from weeks to hours, and achieving near-real-time CDC ingestion.

This talk delivers a live 7-step walkthrough: connector selection, OAuth setup, DLT pipeline config, object selection, Unity Catalog destination mapping, CDC monitoring, and production scheduling.

Takeaways include a cost justification framework demonstrating significant multi-year TCO savings, a migration blueprint from legacy and cloud-native ingestion tools, and governance best practices with Unity Catalog lineage.

Session Speakers

Rishabh Pandey

/Consultant - Databricks Engineer
Celebal Technologies