MGM Enterprise Data Hub: Accelerating Data Delivery With DABS CI/CD Approach
Overview
| Experience | In Person |
|---|---|
| Track | Data Engineering & Streaming |
| Industry | Communications, Media & Entertainment, Travel & Hospitality |
| Technologies | Databricks SQL, Unity Catalog |
| Skill Level | Intermediate |
| DOWNLOAD SESSION SLIDES | |
MGM Resorts is unifying our data estate, leveraging Unity Catalog for centralized governance, access control and cross‑environment consistency across gaming, hospitality and partner ecosystems. We updated our framework with DABS reducing delivery time for gold tables by 67%, enabling automated deployment and validation in under 5 min, cutting deployment code by 75%.
We:
- Enabled isolation and targeted lifecycle management via a multi-bundle architecture with shared utilities in a common wheel
- Improved reliability with a GitHub Actions pipeline for sequential DEV → UAT → PROD deployments, with consistent governance across environments using Unity Catalog
- Freed engineers to focus on business logic through centralized configuration, SCD handling, and Apache Spark™ lifecycle management
Key takeaways:
- CI/CD best practices using standardized databricks.yml
- Higher reliability and faster delivery through a bundle-based pattern
- Metrics and lessons to drive low-risk, predictable releases across the org
Session Speakers
AJAY CHANDER Chevva
/Principle Data Engineer
MGM Resorts International
Jennifer Young
/Vice President of Data
MGM Resorts International
Full Summary
How MGM Resorts scaled 50+ data pipelines with Databricks automation bundles
MGM Resorts is consolidating a decade of data sprawl and retiring legacy platforms into a single enterprise data hub on Azure Databricks. The objective is an end-to-end view of the guest journey, spanning check-in, gaming, shows, dining and spa, so the company can deliver more personalized experiences across properties.
FAQ
Monorepo growth made long-lived branches hard to manage and rollback, which created messy histories and coordination overhead. Trunk-based development simplified flow and, combined with a deployment manifest, provided finer control over what gets promoted without managing multiple release branches.
Production deploys are triggered by tags that follow release//. The workflow compares the new tag to the previous one to detect changed bundles, then consults the domain's deployment manifest to confirm which of those changes are approved for production before deploying.
Shared utilities that multiple domains use, such as helpers for reading inbound tables and writing outbound tables. Centralizing these components prevents duplication and ensures consistent versioning and consumption across domains.
Jobs subscribe to silver tables and run only when upstream data changes. Settings like min-time-between-triggers and a short cool-off window avoid bursty runs when sources update frequently, which keeps serverless utilization and costs in check.
Unity Catalog provides governance and lineage across the enterprise data hub. Jobs execute under service principals for machine-to-machine access, and bundle-level permissions declare which users and groups have access, ensuring consistent, auditable controls across environments.