Agent Evaluations

This hands-on course introduces learners to evaluating, governing, and securing agentic AI systems on Databricks. You'll begin by exploring the motivation for evaluation and governance frameworks and how they connect to the Databricks Data Intelligence Platform. Next, you'll learn how to apply MLflow evaluation metrics and a variety of evaluation techniques, including online evaluation, synthetic evaluation, and building evaluation datasets from MLflow traces. The course then examines how Unity Catalog governance extends to agents' functions, models, and tools, ensuring proper access control, auditability, and compliance. Finally, you'll learn Databricks AI Security Framework (DASF) and how to secure agents with Mosaic AI Gateway, applying guardrails, rate limits, and policy filters to enforce safe and reliable usage.

Note: Hands-on training courses will be updated to reflect the newest product and feature announcements from Data + AI Summit in June 2026. 

Prerequisites

• Familiarity with the Databricks Data Intelligence Platform, including Unity Catalog for governance and model management
• Intermediate Python programming experience
• Familiarity with Generative AI fundamentals, including RAG architectures and their components (chunking, embeddings, vector stores)
• Basic knowledge of MLflow for experiment tracking and model lifecycle management
• General awareness of data governance principles, including access control, data lineage, and data privacy