Learning from Goldman Sachs' Legend Lakehouse for Data Governance
Overview
Experience | In Person |
---|---|
Type | Breakout |
Track | Data Strategy |
Industry | Financial Services |
Technologies | Apache Spark, Apache Iceberg, Unity Catalog |
Skill Level | Intermediate |
Data is the backbone of modern decision-making, but centralizing it is only the tip of the iceberg. Entitlements, secure sharing and just-in-time availability are critical challenges to any large-scale platform. Join Goldman Sachs as we reveal how our Legend Lakehouse, coupled with Databricks, overcomes these hurdles to deliver high-quality, governed data at scale. By leveraging an open table format (Apache Iceberg) and open catalog format (Unity Catalog), we ensure platform interoperability and vendor neutrality. Databricks Unity Catalog then provides a robust entitlement system that aligns with our data contracts, ensuring consistent access control across producer and consumer workspaces. Finally, Legend functions, integrating with Databricks User Defined Functions (UDF), offer real-time data enrichment and secure transformations without exposing raw datasets. Discover how these components unite to streamline analytics, bolster governance and power innovation.
Session Speakers
Tim Smith
/Managing Director
Goldman Sachs