Sponsored by: AWS | Breaking Data Silos with Multi-Engine Catalog Federation

Mastercard moved beyond the "Multi-Catalog" struggle and successfully federated Databricks Unity Catalog to AWS Glue. This allows Apache Spark engines to consume data in Apache Iceberg tables that register in Databricks Unity Catalog and sync seamlessly across AWS services like Amazon EMR, enabling ACID reliability and multi-engine interoperability without vendor lock-in. Mastercard achieved a Single Source of Truth for governance allowing engineers to use the best engine for the job without metadata fragmentation. Mastercard achieved seamless catalog federation, allowing domain teams to choose optimal compute engines. This open-format strategy created a sustainable, future-proof architecture supporting machine learning, streaming, and governed data sharing across the enterprise.