Mastering Change Data Capture With DLT
Overview
Experience | In Person |
---|---|
Type | Breakout |
Track | Data Engineering and Streaming |
Industry | Enterprise Technology |
Technologies | DLT, LakeFlow |
Skill Level | Intermediate |
Duration | 40 min |
Transactional systems are a common source of data for analytics, and Change Data Capture (CDC) offers an efficient way to extract only what’s changed. However, ingesting CDC data into an analytics system comes with challenges, such as handling out-of-order events or maintaining global order across multiple streams. These issues often require complex, stateful stream processing logic.
This session will explore how DLT simplifies CDC ingestion using the Apply Changes function. With Apply Changes, global ordering across multiple change feeds is handled automatically — there is no need to manually manage state or understand advanced streaming concepts like watermarks. It supports both snapshot-based inputs from cloud storage and continuous change feeds from systems like message buses, reducing complexity for common streaming use cases.
Session Speakers
IMAGE COMING SOON
Ray Zhu
/Product Management
Databricks
IMAGE COMING SOON
Jacob Gollub
/Software Engineer
Square