Session

Cielo modernized its data platform and migrated 6,000+ pipelines from EMR to Databricks.

Overview

ExperienceIn Person
TrackData Strategy
IndustryFinancial Services
TechnologiesAI/BI, Databricks SQL, Unity Catalog
Skill LevelAdvanced
Cielo, a leading LATAM payment processor, operated a fragmented analytics landscape across AWS EMR, Oracle Exadata/DW, and decentralized governance. This architecture increased operational overhead, limited lineage and observability, and constrained AI adoption.In this session, we present how Cielo migrated multiple enterprise-grade platforms to a unified Databricks Lakehouse. Over 6,500 pipelines were assessed and over 3,000 legacy jobs eliminated. The migration leveraged Delta Lake, serverless compute, standardized engineering patterns, and Unity Catalog–ready governance.Key takeaways include a ~40% cost reduction in ingestion pipelines, improved reliability and reprocessing, and more than 1,000 users enabled through self-service analytics. We also share practical migration patterns from AWS EMR and Oracle Exadata/DW to Databricks, governance-by-design lessons, and how AI-assisted refactoring accelerated modernization while improving developer productivity.

Session Speakers

Speaker placeholderIMAGE COMING SOON

Gabriel Mochnacs de Arruda

/Chief Data Officer
Cielo S.A. – Instituição de Pagamento