Automated Deployment with Declarative Automation Bundles

This course provides a comprehensive review of DevOps principles and their application to Databricks projects. It begins with an overview of core DevOps, DataOps, continuous integration (CI), continuous deployment (CD), and testing, and explores how these principles can be applied to data engineering pipelines.

The course then focuses on continuous deployment within the CI/CD process, examining tools like the Databricks REST API, SDK, and CLI for project deployment. You will learn about Declarative Automation Bundles (DABs) and how they fit into the CI/CD process. You’ll dive into their key components, folder structure, and how they streamline deployment across various target environments in Databricks. You will also learn how to add variables, modify, validate, deploy, and execute Declarative Automation Bundles for multiple environments with different configurations using the Databricks CLI.

Finally, the course introduces Visual Studio Code as an Interactive Development Environment (IDE) for building, testing, and deploying Declarative Automation Bundles locally, optimizing your development process. The course concludes with an introduction to automating deployment pipelines using GitHub Actions to enhance the CI/CD workflow with Declarative Automation Bundles.

By the end of this course, you will be equipped to automate Databricks project deployments with Declarative Automation Bundles, improving efficiency through DevOps practices.

Note:

1. Databricks Academy is transitioning from video lectures to a more streamlined PDF format with slides and notes for all self-paced courses. Please note that demo videos will still be available in their original format. We would love to hear your thoughts on this change, so please share your feedback through the course survey at the end. Thank you for being a part of our learning community!

2. This course is the fourth in the 'Advanced Data Engineering with Databricks' series.

Skill Level

Professional

Duration

Prerequisites

In this course, the content was developed for participants with these skills/knowledge/abilities:

• Strong knowledge of the Databricks platform, including experience with Databricks Workspaces, Apache Spark, Delta Lake, the Medallion Architecture, Unity Catalog, Delta Live Tables, and Workflows. In particular, knowledge of leveraging Expectations with DLTs.

• Experience in data ingestion and transformation, with proficiency in PySpark for data processing and DataFrame manipulation. Candidates should also have experience writing intermediate-level SQL queries for data analysis and transformation.

• Proficiency in Python programming, including the ability to design and implement functions and classes, and experience with creating, importing, and utilizing Python packages.

• Familiarity with DevOps practices, particularly continuous integration and continuous delivery/deployment (CI/CD) principles.

• A basic understanding of Git version control.

• Prerequisite course DevOps Essentials for Data Engineering Course

Self-Paced

Custom-fit learning paths for data, analytics, and AI roles and career paths through on-demand videos

Customer registration Partner registration

See all our registration options

Registration options

Databricks has a delivery method for wherever you are on your learning journey

Self-Paced

Custom-fit learning paths for data, analytics, and AI roles and career paths through on-demand videos

Instructor-Led

Public and private courses taught by expert instructors across half-day to two-day courses

Blended Learning

Self-paced and weekly instructor-led sessions for every style of learner to optimize course completion and knowledge retention. Go to Subscriptions Catalog tab to purchase

Purchase now

Skills@Scale

Comprehensive training offering for large scale customers that includes learning elements for every style of learning. Inquire with your account executive for details

Upcoming Public Classes

Generative AI Engineer

Get Started with Databricks for Generative AI - Mandarin Chinese

本课程提供 Databricks Data Intelligence Platform 的实践入门，重点介绍用于构建和部署生成式 AI 系统的关键组件与功能。学员将了解 Databricks 如何促进可扩展生成式 AI 解决方案的开发，并浏览 AI Search、Agent Framework 以及 MLflow 的生成式 AI 功能（用于模型追踪与日志记录）等工具。本课程包含构建和评估 Retrieval-Augmented Generation (RAG) 管道、部署生成式 AI 智能体，以及利用评估框架优化性能的实践经验。课程结束时，学员将具备在 Databricks Data Intelligence Platform 上设计、部署和监控常见生成式 AI 应用程序的技能。

Languages Available: English | 日本語 | Português BR | 한국어 | 普通话 | Français | Español

Paid & Subscription

Lab

Onboarding

Omnigent Basics

This course is an introduction to Omnigent, Databricks' open-source meta-harness for AI agents, and how to build, govern, and collaborate on agents with it on the Databricks platform. You'll learn what a harness is and how a meta-harness sits above the agent tools you already use to make them interoperable, and you'll explore Omnigent's three pillars: Combine, Control, and Share. Through interactive lectures and labs, you'll start Omnigent sessions on the Databricks Sandbox, vibe-code and share applications, declare portable YAML agents, enforce stateful contextual policies (including defenses against the lethal trifecta and prompt injection), and work with the built-in Polly and Debby agents. By the end of this course, you'll understand how to coordinate and govern AI agents using Omnigent, Unity Gateway, and the Databricks Sandbox.

Note: For SCORM lecture files, please ensure that you close the SCORM window after completing the content. Do not click the ‘Next Lesson’ button, as doing so may prevent the SCORM module from being marked as complete.

Data Ingestion with Lakeflow Connect - Mandarin Chinese

本课程全面介绍 LakeFlow Connect——一种可扩展且简便的解决方案，用于将来自各种来源的数据摄取到 Databricks 中。您将首先浏览 LakeFlow Connect 连接器的不同类型（标准型和托管型），并学习各种数据摄取技术，包括批处理摄取、增量批处理摄取和流式处理摄取。您还将了解使用 Delta 表和金银铜架构的主要优势。

接下来，您将培养使用 LakeFlow Connect 标准连接器从云对象存储摄取数据的实践技能。这包括使用 CREATE TABLE AS SELECT（CTAS）、COPY INTO 和 Auto Loader 等方法，并重点介绍每种方法的优势和注意事项。您还将学习如何在将数据摄取到 Databricks Data Intelligence Platform 的过程中，向铜层表追加元数据列。课程随后介绍如何使用救援数据列处理与表架构不匹配的记录，以及管理和分析此类数据的策略。您还将探索摄取和展平半结构化 JSON 数据的技术。

此后，您将探索如何使用 LakeFlow Connect 托管连接器执行企业级数据摄取，以引入来自数据库和软件即服务（SaaS）应用程序的数据。课程还将介绍 Partner Connect，作为将合作伙伴工具集成到摄取工作负载中的一种选项。

最后，课程以替代摄取策略作为总结，包括 MERGE INTO 运营以及利用 Databricks Marketplace，为您奠定坚实的基础，以支持现代数据工程用例。

注意：对于 SCORM 课程文件，请确保完成内容后关闭 SCORM 窗口。请勿点击“下一课”按钮，否则可能导致 SCORM 模块无法标记为已完成。

Paid & Subscription

Lab

Associate