Session

Iceberg Table Format Adoption and Unified Metadata Catalog Implementation in Lakehouse Platform

Overview

ExperienceIn Person
TypeBreakout
TrackData Lakehouse Architecture and Implementation
IndustryEnterprise Technology
TechnologiesDelta Lake, Apache Iceberg, Unity Catalog
Skill LevelIntermediate
Duration40 min
DoorDash Data organization actively adopts LakeHouse paradigm. This presentation describes the methodology which allows to migrate the classic Data Warehouse and Data Lake platforms to unified LakeHouse solution.The objective of this effort include Elimination of excessive data movement.Seamless integration and consolidation of the query engine layers, including Snowflake, Databricks, EMR and Trino.Query performance optimization.Abstracting away complexity of underlying storage layers and table formatsStrategic and justified decision on the Unified Metadata catalog used across varios compute platforms

Session Speakers

Sergey Zavgorodni

/Lead Data Engineer
DoorDash

Ruotian Wang

/Software Engineer
Doordash