SESSION

Data Utility Architecture: A Lakehouse Serving the Whole Enterprise

OVERVIEW

EXPERIENCEIn Person
TYPEBreakout
TRACKData Lakehouse Architecture
INDUSTRYEnterprise Technology, Manufacturing, Professional Services
TECHNOLOGIESApache Spark, Delta Lake, Governance
SKILL LEVELIntermediate
DURATION40 min

In 2020, Zhamak Dehghani introduced the Data Mesh concept, uniting data teams but keeping operational and analytical data planes separate. The Data Utility Architecture (DUA), powered by Lakehouse and streaming technologies, breaks down the wall between these planes, uniting all enterprise data. DUA envisions data as a commodity and the pipelines as a utility that serves the whole enterprise regardless of role or persona. It ingests, transforms, and delivers data on demand for any purpose, challenging previous norms that saw operational and analytical data as distinct. Instead, DUA handles all data uniformly, simplifying enterprise data architecture and addressing specific consumer needs with appropriate Service Level Agreements (SLAs), models, and storage. This session will delve into how to reimagine enterprise architecture as a utility using tools like Spark Structured Streaming, Delta tables, and Delta Sharing based on production experience by PwC Canada's Tax DataOps team.

SESSION SPEAKERS

Eric Feunekes

/Data and Generative AI Lead, Tax
PwC