SESSION

Democratize Data Discovery and Data Insight with the Databricks Platform

OVERVIEW

EXPERIENCEIn Person
TYPELightning Talk
TRACKData Governance
INDUSTRYEnterprise Technology
TECHNOLOGIESGovernance
SKILL LEVELIntermediate
DURATION20 min

In the rapidly advancing morden data world, the power of data is undeniable. However, unlocking this power requires data engineers, data producers, and data consumers must collaborate together to build the data trust to understand the entire picture of the data lifecycle. In this talk, we will discuss how democratizing data insights leads to enhanced decision-making, fosters a data-driven culture, and accelerates innovation by allowing more members of the organization to leverage data in their roles. This will include sharing a few updates in the following areas:

  • Catalog Explorer: how we build the smart and intuitive modernized data portal for Databricks Unity Catalog.
  • Data Lineage: How do we build the end-to-end lakehouse lineage, including ML lineage for Unity Catalog?
  • Data insight: how we build the data insight, including leveraging AI / LLM to generate auto-documentation, surfacing data popularity, and other trust signals.

SESSION SPEAKERS

Tao Feng

/Sr Staff Software Engineer
Databricks

Aly Hirani

/Software Engineer
Databricks