Session
Iceberg Table Format Adoption and Unified Metadata Catalog Implementation in Lakehouse Platform
Overview
Experience | In Person |
---|---|
Type | Breakout |
Track | Data Lakehouse Architecture and Implementation |
Industry | Enterprise Technology |
Technologies | Delta Lake, Apache Iceberg, Unity Catalog |
Skill Level | Intermediate |
Duration | 40 min |
DoorDash Data organization actively adopts LakeHouse paradigm. This presentation describes the methodology which allows to migrate the classic Data Warehouse and Data Lake platforms to unified LakeHouse solution.The objective of this effort include Elimination of excessive data movement.Seamless integration and consolidation of the query engine layers, including Snowflake, Databricks, EMR and Trino.Query performance optimization.Abstracting away complexity of underlying storage layers and table formatsStrategic and justified decision on the Unified Metadata catalog used across varios compute platforms
Session Speakers
Sergey Zavgorodni
/Lead Data Engineer
DoorDash
Ruotian Wang
/Software Engineer
Doordash