Cloud Migration and Cost Optimization in the Databricks Lakehouse

Allos had a chaotic data environment, and when our company decided to perform a migration from Azure to Google Cloud, we thought that it would be a great time to revolutionize our data environment, so we used Delta Lake and Unity Catalog and integrated these technologies with GitLab CI/CD to automate our deploying process and define development standards. The outcome of this project was a full cloud migration in less than 6 months and a cost reduction of more than 90% in the whole data environment ($48.000/mo to $5.000/mo). Besides that, now we benefit from all the advantages of a data lakehouse, including time travel and unified governance.