Session
Unity Catalog Implementation & Evolution at Edward Jones
Overview
Experience | In Person |
---|---|
Type | Lightning Talk |
Track | Data and AI Governance |
Industry | Financial Services |
Technologies | Delta Lake, Unity Catalog |
Skill Level | Intermediate |
Duration | 20 min |
Analytics Hub is a product which serves as the central repository for analytics and reporting data across Edward Jones.
Analytics Hub followed the UC design aligning to the Medallion architecture:
- Staged catalog for landing and maintaining the raw datasets in its original format
- Two catalogs for enterprise and point solution aligning datasets which are guard railed by the enterprise and business taxonomies
- Application catalog, with portfolio & product hierarchical alignment of applications with each of it having autonomous UC Schemas
We have good amount of data that are migrated to Analytics Hub and our user base is growing exponentially. This comes with next level of challenge of getting ready for DR.
In a nutshell, we will talk about:
- Inception of Unity Catalog
- Cohabit with Edwardjones ecosystems like Collibra, Immuta, Snowflake, ingestion patterns
- Lessons learnt about the brownfield
- Strategy for new Greenfield and migration
- DR readiness
Session Speakers
Dattatri Rao
/Technical Architect
Edward Jones