Agent Evaluation on Databricks

This course teaches students how to systematically evaluate AI agents using MLflow's evaluation framework, addressing the unique challenges of non-deterministic AI systems that traditional software testing cannot handle. Students learn to implement various evaluation approaches including built-in judges for common criteria like correctness and safety, guideline judges for business-specific requirements, and custom judges for specialized needs. The course covers both offline evaluation using curated datasets and online production monitoring, with hands-on experience using MLflow's tracing capabilities to understand agent execution patterns and collect human feedback from different stakeholder types. Through practical demonstrations and labs, students develop skills in creating evaluation workflows that drive continuous quality improvements throughout the AI agent development lifecycle.

Skill Level

Associate

Duration

Prerequisites

In this course, the content was developed for participants with these skills/knowledge/abilities:

• Intermediate Python programming experience

• Basic SQL knowledge for querying and creating functions

• Familiarity with Databricks Data Intelligence Platform

• Understanding of Unity Catalog concepts including catalogs and schemas

• Basic understanding of large language models (LLMs) and prompt engineering

• Basic knowledge of MLflow

Self-Paced

Custom-fit learning paths for data, analytics, and AI roles and career paths through on-demand videos

Customer registration Partner registration

See all our registration options

Registration options

Databricks has a delivery method for wherever you are on your learning journey

Self-Paced

Custom-fit learning paths for data, analytics, and AI roles and career paths through on-demand videos

Instructor-Led

Public and private courses taught by expert instructors across half-day to two-day courses

Blended Learning

Self-paced and weekly instructor-led sessions for every style of learner to optimize course completion and knowledge retention. Go to Subscriptions Catalog tab to purchase

Purchase now

Skills@Scale

Comprehensive training offering for large scale customers that includes learning elements for every style of learning. Inquire with your account executive for details

Upcoming Public Classes

Get Started with Databricks for Machine Learning - Mandarin Chinese

在本课程中，您将培养使用 Databricks Data Intelligence Platform 执行机器学习工作流并支持数据科学工作负载所需的基础技能。您将从机器学习从业者的视角探索该平台，内容涵盖使用 Mosaic AI Feature Engineering 构建和管理特征、使用 MLflow 进行端到端模型生命周期管理，以及使用 Lakeflow Jobs 进行管道编排等主题。此外，您还将学习如何使用 Databricks AI Model Serving 进行实时模型推理，并通过 Genie Code - Data Science Agent Mode 体验 Databricks 透明、对话式的模型开发方式——在其中，您可以使用自然语言提示，直接在笔记本中生成、运行并迭代优化可执行的 ML 工作流。本课程包含由讲师主导的演示，并以一个全面的实验作为收尾，巩固贯穿全程所讲解的概念。

Languages Available: English | 日本語 | Português BR | 한국어

Get Started with Databricks for Data Engineering

This course introduces the foundational skills needed to perform a basic data engineering workflow on the Databricks Data Intelligence Platform. You will explore the workspace, work with Unity Catalog, and learn the day-to-day building blocks a data engineer uses on Databricks.

The course follows a hands-on path. You start by getting oriented in the workspace, then work through paired Demo and Lab notebooks for each topic. Demos walk you through a concept with the instructor or a guided notebook.

The content covers:

• The Databricks Data Intelligence Platform and how the Databricks workspace, Unity Catalog, and notebooks fit together.

• Creating and managing Delta Lake tables.

• Modifying data with INSERT, UPDATE, and DELETE.

• Exploring version history and time travel on Delta tables.

• Ingesting data with Lakeflow Connect options: CTAS, the Upload UI, and COPY INTO.

• Building a Medallion Architecture pipeline that transforms data through Bronze, Silver, and Gold layers.

• Automating pipelines with Lakeflow Jobs.

• (Bonus) Building a declarative pipeline with Spark Declarative Pipelines.

AI/BI for Data Analysts

This course teaches data analysts how to design, build, publish, and operate AI/BI Dashboards in Databricks. AI/BI Dashboards combine governed Unity Catalog data with interactive visualizations, filters, and Genie integration so business users can explore answers without writing code.

The course follows a single end-to-end build. You start with source tables in Unity Catalog and finish with a published, monitored multi-page dashboard. Along the way you learn how dashboards fit into the broader Databricks AI/BI product family and where Genie, datasets, visualizations, and filters each fit in the workflow.

The content covers:

• AI/BI Dashboard fundamentals and how they relate to Genie and the rest of the Databricks platform.

• Exploring source data in Unity Catalog and designing reusable dashboard datasets with SQL.

• Authoring visualizations (KPIs, trends, breakdowns) and laying out a clean multi-page dashboard.

• Using Genie Code to draft SQL, charts, and filters from natural language prompts.

• Adding filters to make dashboards interactive and responsive to viewer questions.

• Publishing, sharing, and managing permissions so the right people can view and edit the dashboard.

• Running the dashboard in production with scheduled refresh, caching, and usage monitoring.

Note: For SCORM lecture files, please ensure that you close the SCORM window after completing the content. Do not click the ‘Next Lesson’ button, as doing so may prevent the SCORM module from being marked as complete.

Paid & Subscription

Lab

Associate