Delivering Portability to Open Data Lakes with Delta Lake UniForm


TYPELightning Talk
TRACKData Lakehouse Architecture
INDUSTRYEnterprise Technology, Manufacturing
TECHNOLOGIESData Sharing, Apache Spark, Delta Lake
SKILL LEVELIntermediate

As data volumes and users rapidly scale, data lakes encounter major challenges around reliability, performance, and governance. Delta Lake UniForm (Universal Format) helps address these pain points on multiple open data lake environments such as Delta Lake, Apache Iceberg, and Apache Hudi. This talk will demonstrate how Delta Lake UniForm enables seamless and unifying access to multiple open data lakes while optimizing workloads. We also deeply dive into key technology behind the UniForm that improves portability, reliability, and query performance. Through live demos, we showcase scaling a cloud-based data lake from terabytes to petabytes while maintaining ACID transactions, audit history, and so on. We deliver Delta Lake UniForm best practices to future-proof their own expanding data lakes. The UniForm capabilities make data lakes more accessible to diverse users.


Tomohiro Tanaka

/Senior Cloud Support Engineer
Amazon Web Services