Deploy Workloads with Lakeflow Jobs

Deploy Workloads with Lakeflow Jobs course teaches how to orchestrate and automate data, analytics, and AI workflows using Lakeflow Jobs as a unified orchestration platform within the Databricks ecosystem.

⇾ You will learn to design and implement data workloads using Directed Acyclic Graphs (DAGs), configure various scheduling options, and implement advanced workflow features such as conditional task execution, run-if dependencies, and for each loops.

⇾ The course covers best practices for creating robust, production-ready pipelines with proper compute selection, modular orchestration, error handling techniques, and fault-tolerant design-all natively integrated within the Databricks Data Intelligence Platform.

Note: Databricks Academy is transitioning from video lectures to a more streamlined PDF format with slides and notes for all self-paced courses. Please note that demo videos will still be available in their original format. We would love to hear your thoughts on this change, so please share your feedback through the course survey at the end. Thank you for being a part of our learning community!

Skill Level

Associate

Duration

Prerequisites

⇾ Completion of the course "Get Started with Databricks for Data Engineering", or a solid understanding of the Databricks Data Intelligence Platform

⇾ Basic Understanding of topics like navigating a Databricks Workspace, Apache Spark, Delta Lake, Medallion Architecture, and Unity Catalog.

⇾ Familiarity with python/pyspark

⇾ Experience in writing intermediate-level SQL queries.

Self-Paced

Custom-fit learning paths for data, analytics, and AI roles and career paths through on-demand videos

Customer registration Partner registration

See all our registration options

Registration options

Databricks has a delivery method for wherever you are on your learning journey

Self-Paced

Custom-fit learning paths for data, analytics, and AI roles and career paths through on-demand videos

Instructor-Led

Public and private courses taught by expert instructors across half-day to two-day courses

Blended Learning

Self-paced and weekly instructor-led sessions for every style of learner to optimize course completion and knowledge retention. Go to Subscriptions Catalog tab to purchase

Purchase now

Skills@Scale

Comprehensive training offering for large scale customers that includes learning elements for every style of learning. Inquire with your account executive for details

Upcoming Public Classes

Data Engineer

Data Interoperability with Unity Catalog

This course explores how to achieve seamless data interoperability between systems using Databricks Unity Catalog as a unified governance layer, using both Delta and Iceberg open table formats. Participants will learn to implement, manage and optimize a data environment where tables can be accessed by both Delta and Iceberg clients, enabling cross platform analytics and eliminating data silos.

Advanced Techniques with Spark Declarative Pipelines

This course explores Databricks' Lakeflow Spark Declarative Pipelines (SDP) for building production-grade streaming pipelines. You will learn advanced design patterns, robust data quality enforcement, and cross-platform integration essential for real-world lakehouse engineering.

Throughout the course, you will dive into modern data ingestion and processing techniques, mastering tools like Liquid Clustering for layout optimization and the Multiplex Streaming pattern for mixed-schema events. By the end of the modules, you will know how to confidently handle schema evolution, automate Change Data Capture (CDC), and ensure data integrity.

Through lectures and hands-on demos, you will:

• Build multi-flow pipelines to ingest multi-source data into a unified Bronze table.

• Apply Liquid Clustering and Data Quality Expectations across Silver and Gold layers.

• Implement the Multiplex pattern with Iceberg UniForm for cross-platform data access.

• Automate SCD Type 2 history tracking using AUTO CDC INTO.

• Design zero-data-loss quarantine pipelines to audit and manage invalid records.

Note:

1. This course is the first in the 'Advanced Data Engineering with Databricks' series.

2. For SCORM lecture files, please ensure that you close the SCORM window after completing the content. Do not click the ‘Next Lesson’ button, as doing so may prevent the SCORM module from being marked as complete.

Data Modeling Strategies

This course offers a deep dive into designing data models within the Databricks Lakehouse environment, and understanding the data products lifecycle. Participants will learn to align business requirements with data organization and model design leveraging Delta Lake and Unity Catalog for defining data architectures, and techniques for data integration and sharing.

Note: The course includes practice labs that the learners should perform after going through the entire course.

Languages Available: English | 日本語 | Português BR | 한국어

Paid & Subscription

Lab

Associate