Get Started with SQL Analytics and BI on Databricks

This course teaches students how to systematically evaluate AI agents using MLflow's evaluation framework, addressing the unique challenges of non-deterministic AI systems that traditional software testing cannot handle. Students learn to implement various evaluation approaches including built-in judges for common criteria like correctness and safety, guideline judges for business-specific requirements, and custom judges for specialized needs. The course covers both offline evaluation using curated datasets and online production monitoring, with hands-on experience using MLflow's tracing capabilities to understand agent execution patterns and collect human feedback from different stakeholder types. Through practical demonstrations and labs, students develop skills in creating evaluation workflows that drive continuous quality improvements throughout the AI agent development lifecycle.

In this course, you will learn how to integrate a Databricks App with Lakebase, Databricks' fully managed, cloud-native PostgreSQL OLTP database. This course assumes some familiarity with Databricks Apps and Lakebase and will therefore not cover the basic of those pieces; instead we'll focus on integrating the two to deliver an app that performs basic data collection and captures it in a Lakebase table. An overview of PostgreSQL and safety best practices will be emphasized.

Note: For SCORM lecture files, please ensure that you close the SCORM window after completing the content. Do not click the ‘Next Lesson’ button, as doing so may prevent the SCORM module from being marked as complete.

This course offers a practical introduction to the Mosaic AI platform, focusing on its key components and features for building and deploying generative AI systems. Participants will learn how Databricks facilitates the development of scalable generative AI solutions and explore Mosaic AI tools such as Vector Search, the Agent Framework, and MLflow's generative AI capabilities for model tracking and logging. By the end of the course, learners will be equipped with the skills to design, deploy, and monitor common generative AI applications using Mosaic AI.

Languages Available: English | 日本語 | Português BR | 한국어