Session

Build and Deploy Databricks Projects Using Automation Bundles and Genie ZeroOps

Overview

ExperienceIn Person
TrackData Engineering & Streaming
IndustryHealthcare & Life Sciences, Financial Services, Transportation
TechnologiesGenie
Skill LevelBeginner
DOWNLOAD SESSION SLIDES

Modern data engineering teams need more than deployment automation. They need projects that are source-controlled, governed, and reviewable, so both humans and AI agents can safely build, deploy, and operate them at scale. Declarative Automation Bundles (DABs) provide that foundation. By expressing your entire Databricks project as source code, DABs not only enable scalable processes for human developers, but provide the exact structured interface AI agents need to interact with and manage your workflows. In this session, we will show how to use DABs in practice, including best practices and the latest authoring workflows across Genie Code, local IDEs, and the Workspace UI. We will also introduce Deployment History, a new service to track exactly who deployed what, when, and with what result. Building on this foundation, we will share a first look at our Genie Code powered product that monitors jobs, pipelines, tables, and other data assets for failures and optimization opportunities. It explains the root cause, proposes a fix, and validates it in a sandboxed environment. Today, data engineers spend 50%+ of their time maintaining data assets. As agentic tools make code generation faster and more broadly available, removing that operational bottleneck is more urgent than ever.

 

Session Speakers

Kristóf Molnár

/Sr. Staff Product Manager
Databricks

Lennart Kats

/Principal Engineer
Databricks

Full Summary

Genie ZeroOps: how agents and Declarative Automation Bundles reshape data pipeline operations

Many data teams spend most of their time on break-fix work, triaging noisy alerts and chasing silent data regressions.

FAQ


No. The agent proposes fixes and can open a pull request, but all changes go through your existing CI, review, testing, and approval processes. It operates on a copy of the code and validates hypotheses in a read-only sandbox rather than modifying production data.

DABs are strongly recommended. Declaring jobs, pipelines, tests, and environments as code gives the agent consistent context to diagnose failures accurately and propose safe, reviewable changes.

It prioritizes assets you mark as important, learns from which insights users engage with, and groups downstream symptoms under a single root cause. The inbox focuses attention on incidents that matter and explains the reasoning behind each one.

The agent can validate in a sandbox with read-only access to production. It issues SQL reads to confirm schemas or data states, then prepares a pull request so humans can review and approve the change.

Genie ZeroOps is expected to enter private preview around the end of July, and deployment history is expected to enter private preview before the end of summer. Enrollment is handled through the interest form referenced in the session.