Delta-Kernel-RS: Unparalleled Interoperability Across Query Engines
Overview
Experience | In Person |
---|---|
Type | Breakout |
Track | Data Lakehouse Architecture and Implementation |
Industry | Enterprise Technology |
Technologies | Delta Lake, Unity Catalog |
Skill Level | Intermediate |
Duration | 40 min |
Join us as we introduce Delta-Kernel-RS, a new Rust implementation of the Delta Lake protocol designed for unparalleled interoperability across query engines. In this session, we will explore how maintaining a native implementation of the Delta specification — with native C and C++ FFI support — can deliver consistent benefits across diverse data processing systems, eliminating the need for repetitive, engine-specific reimplementations. We will dive deep into a real-world case study where a query engine harnessed Delta-Kernel-RS to unlock significant data skipping improvements — enhancements achieved “for free” by leveraging the kernel. Attendees will gain insights into the architectural decisions, interoperability strategies and the practical impact of this innovation on performance and development efficiency in modern data ecosystems.
Session Speakers
IMAGE COMING SOON
Zachary Schuermann
/Software Engineer/Ski bum
databricks
Robert Pack
/Staff Developer Advocate
Databricks