New Capability Overview: Conditional Execution in Workflows

Course Description

Conditional Execution in Workflows is a new 2023 offering from Databricks that allows Databricks Workflows to execute tasks conditionally, based on the state of its dependencies. Two new capabilities work together to achieve this: the “Run if” condition and the “if/else” task 

Previously, a task in a job would only run if all of its dependencies were successful. With the “Run if“ condition, tasks can be configured to execute or be excluded based on the success or failure of upstream tasks. With the new “if/else” task, you can choose tasks to execute based on the result of a condition expressed using task values, trigger information, or parameters. 

The content contained in this course aims to serve as an introduction to Conditional Execution in Workflows on the Databricks Data Intelligence Platform.  

Course Objectives

After consuming the content inside of this learning pathway, you should be able to: 

• Explain fundamental concepts about the impact that Conditional Execution in Workflows has on the Databricks Data Intelligence Platform. 

• Follow-along with a demonstration of how Conditional Execution in Workflows can be used to provide value to common/important use cases.

• Perform hands-on activities using Conditional Execution in Workflows that showcase its most important features to the Databricks Data Intelligence Platform. 

Prerequisites

At a minimum, you should be familiar with the following before attempting to take this content:

• Fundamental concepts about the Databricks Data Intelligence Platform

• Intermediate level-knowledge of Databricks Workflows (consider completing the Deploy Workloads with Databricks Workflows course)

Technical considerations

Please keep these technical considerations in mind as you go through this content (particularly if you plan on following along with demos and completing lab exercises): 

• Databricks Runtime: This course was designed to work with DBR 12.2 LTS. Please use this DBR when working through this course. 

• Feature access: This course requires access to “Run if” condition and “If/Else condition task” in Databricks Workflows

Last course update

• November 28, 2023