Azure Databricks Cloud Integration Demo
The Azure Databricks Lakehouse Platform gives you the best of data lakes and data warehouses, on a simple, open and collaborative platform that securely integrates with your existing Azure services. In this demo, we cover several of the most common Azure Databricks integrations, including Azure Data Lake Storage (ADLS), Azure Data Factory (ADF), Azure IoT Hub, Azure Synapse Analytics, Power BI and more.
Dive deeper into Azure Databricks
Video transcript
Intro to Lakehouse on Azure Databricks
Azure Databricks is the jointly developed Data + AI service from Databricks and Microsoft for data engineering, data science, analytics and machine learning. The Azure Databricks Lakehouse Platform gives you the best of data lakes and data warehouses, on a simple, open and collaborative platform that securely integrates with your existing Azure services.
Batch ETL pipelines with Azure Data Factory and Azure Databricks
Orchestrating data pipelines for extract, transform and load (ETL) operations is easier than ever thanks to the integration between Azure Databricks and Azure Data Factory. In this demo, we’ll be working with power output and maintenance system records from a fleet management system used to maintain wind turbines across the United States. We’ve created an Azure Data Factory pipeline that uses COPY commands to incrementally ingest data from the fleet management system in SQL server to our landing zone in Azure Data Lake Storage Gen2. Data Factory then executes a Databricks notebook to process the data using Databricks Auto Loader, and saves it into Delta Lake tables.
