CI/CD for Databricks: Advanced Asset Bundles and GitHub Actions
Overview
Experience | In Person |
---|---|
Type | Breakout |
Track | Data Engineering and Streaming |
Industry | Enterprise Technology, Health and Life Sciences, Financial Services |
Technologies | Apache Spark, Databricks Workflows, DLT |
Skill Level | Intermediate |
Duration | 60 min |
Databricks Asset Bundles (DABs) provide a way to use the command line to deploy and run a set of Databricks assets — like notebooks, Python code, DLT pipelines and workflows. To automate deployments, you create a deployment pipeline that uses the power of DABs along with other validation steps to ensure high quality deployments.
In this session you will learn how to automate CI/CD processes for Databricks while following best practices to keep deployments easy to scale and maintain. After a brief explanation of why Databricks Asset Bundles are a good option for CI/CD, we will walk through a working project including advanced variables, target-specific overrides, linting, integration testing and automatic deployment upon code review approval. You will leave the session clear on how to build your first GitHub Action using DABs.
Session Speakers
Dustin Vannoy
/Sr. Specialist Solutions Architect
Databricks