Agent Evaluations

This hands-on course introduces learners to evaluating, governing, and securing agentic AI systems on Databricks. You'll begin by exploring the motivation for evaluation and governance frameworks and how they connect to the Databricks Data Intelligence Platform. Next, you'll learn how to apply MLflow evaluation metrics and a variety of evaluation techniques, including online evaluation, synthetic evaluation, and building evaluation datasets from MLflow traces. The course then examines how Unity Catalog governance extends to agents' functions, models, and tools, ensuring proper access control, auditability, and compliance. Finally, you'll learn Databricks AI Security Framework (DASF) and how to secure agents with Mosaic AI Gateway, applying guardrails, rate limits, and policy filters to enforce safe and reliable usage.