Skip to main content

Agent Evaluation on Databricks - Mandarin Chinese

本课程教授学生如何使用 MLflow 的评估框架对 AI 智能体进行系统性评估,以应对传统软件测试无法处理的非确定性 AI 系统所带来的独特挑战。学生将评估一个已注册在 Unity Catalog 中并通过 Unity Gateway 调用其模型的共享工具调用智能体,使用内置评判器处理正确性和安全性等常见标准,使用准则评判器满足特定业务需求,并使用自定义评判器应对专项需求。本课程涵盖使用精选数据集进行离线评估和在线生产监控,包括在 Unity Catalog 中存储 MLflow Traces。学生将使用 MLflow Tracing 了解智能体的执行模式,并收集来自开发人员和主题专家的人工反馈,这些专家通过 Databricks App 对智能体进行审查。通过实践演示和实验,学生将掌握创建评估 Workflows 的技能,从而在 AI 智能体开发生命周期中持续推动质量提升。



注意:Databricks Academy 正在转向在 Databricks 环境中使用基于笔记本的课堂授课形式,并将不再使用幻灯片进行讲授。您可以在 Vocareum 实验环境中访问课程讲义笔记本。


Languages Available: English | 日本語 | Português BR | 한국어 | Français | Español | 普通话

Skill Level
Associate
Duration
4h
Prerequisites
本课程内容面向具备以下技能、知识和能力的学员:  

• 具备中级 Python 编程经验

• 具备基本 SQL 知识,能够进行查询和创建函数

• 熟悉 Databricks Data Intelligence Platform

• 了解 Unity Catalog 概念,包括目录和架构

• 基本了解大型语言模型(LLMs)及提示工程

• 具备 MLflow 基础知识

Outline

1. AI 智能体评估基础

• 评估 AI 智能体面临的挑战

• 演示:智能体设置

• MLflow 评估框架


2. 内置评判器和准则评判器

• 内置评判器

• 演示:使用 MLflow 内置评判器

• 准则评判器

• 演示:使用 MLflow 的准则评判器

• 自定义评判器和反馈

• 演示:使用 MLflow 的自定义评判器

• 实验:应用智能体评估


3. 自定义评判器和人工反馈

• 离线与在线评估策略

• 实验:使用 MLflow 收集开发人员和主题专家(SME)的反馈

Public Class Registration

If your company has purchased success credits or has a learning subscription, please fill out the Training Request form. Otherwise, you can register below.

Private Class Request

If your company is interested in private training, please submit a request.

See all our registration options

Registration options

Databricks has a delivery method for wherever you are on your learning journey

Runtime

Self-Paced

Custom-fit learning paths for data, analytics, and AI roles and career paths through on-demand videos

Register now

Instructors

Instructor-Led

Public and private courses taught by expert instructors across half-day to two-day courses

Register now

Learning

Blended Learning

Self-paced and weekly instructor-led sessions for every style of learner to optimize course completion and knowledge retention. Go to Subscriptions Catalog tab to purchase

Purchase now

Scale

Skills@Scale

Comprehensive training offering for large scale customers that includes learning elements for every style of learning. Inquire with your account executive for details

Upcoming Public Classes

Questions?

If you have any questions, please refer to our Frequently Asked Questions page.