Generative AI Engineering with Databricks

This course is aimed at data scientists, machine learning engineers, and other data practitioners who want to build generative AI applications using the latest and most popular frameworks and Databricks capabilities. 

Note: Databricks Academy is transitioning to a notebook-based format for classroom sessions within the Databricks environment, discontinuing the use of slide decks for lectures. You can access the lecture notebooks in the Vocareum lab environment.

Below, we describe each of the four, four-hour modules included in this course.

Building RAG Agents with Agent Bricks: This course provides hands-on training for building retrieval agents using Databricks Agent Bricks. Participants will learn to explore and query Knowledge Assistants, parse unstructured documents into structured data using AI Functions, chunk text for semantic retrieval, build Vector Search indexes, and create production-ready Knowledge Assistants backed by multiple knowledge sources.

Building Agentic Applications on Databricks: This course teaches students how to build production-grade agentic applications on Databricks. Students learn to create governed agent tools using Unity Catalog and MCP, build single and multi-agent systems with the OpenAI Agents SDK, and leverage Agent Bricks and Genie for knowledge-assistant use cases orchestrated with a supervisor agent. The course covers the full progression from tool prototyping to production deployment, with hands-on experience using MLflow tracing to observe agent execution.

Agent Evaluation on Databricks: This course teaches students how to systematically evaluate AI agents using MLflow's evaluation framework, addressing the unique challenges of non-deterministic AI systems that traditional software testing cannot handle. Students learn to implement various evaluation approaches including built-in judges for common criteria like correctness and safety, guideline judges for business-specific requirements, and custom judges for specialized needs. The course covers both offline evaluation using curated datasets and online production monitoring, with hands-on experience using MLflow's tracing capabilities to understand agent execution patterns and collect human feedback from different stakeholder types. Through practical demonstrations and labs, students develop skills in creating evaluation workflows that drive continuous quality improvements throughout the AI agent development lifecycle.

Deploying and Monitoring Agent Applications on Databricks: This course covers the end-to-end lifecycle for deploying and monitoring generative AI agents on Databricks. Participants will learn how to deploy agents as Databricks Apps using Declarative Automation Bundles (DABs), integrate tools via the Model Context Protocol (MCP), instrument agents with MLflow Tracing, and evaluate production quality using scorers, multi-turn judges, and online evaluation. Through hands-on demos and labs, participants will gain practical experience building, observing, and monitoring production-grade AI agents on the Databricks platform.