Session

Read-Time CDF in Delta Lake

Overview

ExperienceIn Person
TrackData Engineering & Streaming
IndustryEnterprise Technology
TechnologiesDatabricks SQL, Lakeflow
Skill LevelIntermediate

Traditionally, enabling Change Data Feed (CDF) in Delta Lake incurs a "write tax"—increasing storage costs and latency to materialize changes during ingestion.In this session, we introduce Read-Time CDF, a new architecture that unlocks zero-overhead writes by deferring change computation to query time. By leveraging the new unified CDC interface in Spark Data Source V2 and Delta’s Row Tracking, users can now query row-level changes without ever explicitly enabling delta.enableChangeFeed.

Session Speakers

Speaker placeholderIMAGE COMING SOON

Gengliang Wang

/Staff Software Engineer
Databricks

Speaker placeholderIMAGE COMING SOON

Johan Lasperas

/Staff Software Engineer
Databricks