Skip to main content

Agent Evaluation on Databricks

This course teaches students how to systematically evaluate AI agents using MLflow's evaluation framework, addressing the unique challenges of non-deterministic AI systems that traditional software testing cannot handle. Students learn to implement various evaluation approaches including built-in judges for common criteria like correctness and safety, guideline judges for business-specific requirements, and custom judges for specialized needs. The course covers both offline evaluation using curated datasets and online production monitoring, with hands-on experience using MLflow's tracing capabilities to understand agent execution patterns and collect human feedback from different stakeholder types. Through practical demonstrations and labs, students develop skills in creating evaluation workflows that drive continuous quality improvements throughout the AI agent development lifecycle.


Note: For SCORM lecture files, please ensure that you close the SCORM window after completing the content. Do not click the ‘Next Lesson’ button, as doing so may prevent the SCORM module from being marked as complete.

Skill Level
Associate
Duration
3h
Prerequisites

In this course, the content was developed for participants with these skills/knowledge/abilities:  

• Intermediate Python programming experience

• Basic SQL knowledge for querying and creating functions

• Familiarity with Databricks Data Intelligence Platform

• Understanding of Unity Catalog concepts including catalogs and schemas

• Basic understanding of large language models (LLMs) and prompt engineering

• Basic knowledge of MLflow

Self-Paced

Custom-fit learning paths for data, analytics, and AI roles and career paths through on-demand videos

See all our registration options

Registration options

Databricks has a delivery method for wherever you are on your learning journey

Runtime

Self-Paced

Custom-fit learning paths for data, analytics, and AI roles and career paths through on-demand videos

Register now

Instructors

Instructor-Led

Public and private courses taught by expert instructors across half-day to two-day courses

Register now

Learning

Blended Learning

Self-paced and weekly instructor-led sessions for every style of learner to optimize course completion and knowledge retention. Go to Subscriptions Catalog tab to purchase

Purchase now

Scale

Skills@Scale

Comprehensive training offering for large scale customers that includes learning elements for every style of learning. Inquire with your account executive for details

Upcoming Public Classes

Get Started with Lakebase - Mandarin Chinese

这门入门课程介绍 Databricks Lakebase,这是一项内置于 Databricks Data Intelligence Platform 的完全托管的 PostgreSQL 服务,它让运营型(OLTP)和分析型(OLAP)工作负载更加紧密地结合在一起。

课程以一节概念讲座开始,对比 OLTP 和 OLAP 系统,讲解它们不同的性能特征、存储模型和典型用例。你还将探讨组织在维护相互独立的事务型数据库和分析平台时所面临的挑战,包括数据移动、延迟和架构复杂性。

随后你将学习 Databricks Lakebase 如何通过提供一个与 Databricks Lakehouse 直接集成、PostgreSQL 兼容的运营型数据库来应对这些挑战,使运营型应用和分析能够在一个统一平台内协同工作。

通过动手实验,你将:

使用 autoscaling compute 创建并探索一个 Lakebase 项目

• 浏览 Lakebase UI,包括 branching、监控和配置设置

• 使用 Lakebase SQL Editor 创建并查询表

• 使用 Lakehouse Federation 和外部目录从 Databricks 查询 Lakebase 数据

• 通过将 Delta 表同步到 Lakebase 来执行 Reverse ETL

• 从 Python 连接到 Lakebase 并执行基本的 CRUD 操作

这是一门入门(Get Started)课程,因此重点在于理解使用 Lakebase 的核心概念和基本工作流。在 Lakebase 之上构建完整的生产应用不在本课程的范围之内。

注意:对于 SCORM 课程文件,请在完成内容后关闭 SCORM 窗口。请勿点击“下一课”按钮,否则可能导致 SCORM 模块无法被标记为已完成。

Paid & Subscription
3h
Onboarding

Questions?

If you have any questions, please refer to our Frequently Asked Questions page.