The Road to Delta 5.0

Marking its ninth anniversary this year, Delta Lake has grown significantly since its 2017 debut, spanning four major versions and 33 individual releases. As a premier ACID-compliant open table format globally, it sees 40M+ downloads per month and powers the daily processing of hundreds of exabytes of data. While the current roadmap includes the recent launch of Delta 4.3, the project is undergoing foundational shifts as it moves toward Delta 5.0. This session explores these paradigm shifts through two primary lenses: Evolution and Unification.

Key topics will include:

• The transition of Delta into a catalog-first table format.
• Modernizing Delta on Spark by leveraging Data Source V2 APIs.
• The convergence of the Delta and Iceberg formats.
• Harmonizing the Delta Kernel across Java and Rust implementations.