Build Data Pipelines with Lakeflow Spark Declarative Pipelines

This hands-on course introduces users to the essential concepts and skills needed to build data pipelines using Lakeflow Spark Declarative Pipelines (SDP) in Databricks for incremental batch or streaming ingestion and processing through multiple streaming tables and materialized views. Designed for data engineers new to Spark Declarative Pipelines, the course provides a comprehensive overview of core components such as incremental data processing, streaming tables, materialized views, and temporary views, highlighting their specific purposes and differences.

Note: Hands-on training courses will be updated to reflect the newest product and feature announcements from Data + AI Summit in June 2026. 

Prerequisites

• Familiarity with the Databricks Data Intelligence Platform, including workspaces, Apache Spark, Delta Lake, Medallion Architecture, Lakeflow Jobs, and Unity Catalog
• Experience ingesting raw data into Delta tables using the read_files SQL function (CSV, JSON, TXT, Parquet)
• Intermediate SQL proficiency, including intermediate-level queries and joins
• Understanding of ETL concepts and batch/streaming workflows