Apache Spark™ Programming with Databricks

This course is designed to introduce three primary machine learning deployment strategies and illustrate the implementation of each strategy on Databricks. Following an exploration of the fundamentals of model deployment, the course delves into batch inference, offering hands-on demonstrations and labs for utilizing a model in batch inference scenarios, along with considerations for performance optimization. The second part of the course comprehensively covers pipeline deployment, while the final segment focuses on real-time deployment. Participants will engage in hands-on demonstrations and labs, deploying models with Model Serving and utilizing the serving endpoint for real-time inference.

This course will guide participants through an exploration of machine learning operations on Databricks. This will begin with an introduction to modern machine learning operations as a combination of DevOps, DataOps, and ModelOps, including an overview of each component. Next, the course will cover the Databricks-recommended architectures and solutions for machine learning operations. This will focus on capabilities like MLOps Stacks, MLflow, Model Serving, Unity Catalog, and Lakehouse Monitoring. Finally, the final section of the course will be a practical walkthrough of a simple machine learning operations project using these capabilities, including: an initial development phase, a staging phase, and a production stage. By the end of this course, learners will be familiar with the basic end-to-end machine learning operations workflow on Databricks.

In this Data Governance with Unity Catalog session, you'll learn concepts and watch demos that showcase workflows using Unity Catalog - Databricks' solution to data governance. We'll start off with a brief introduction to Unity Catalog, discuss fundamental data governance concepts and then dive into a variety of topics including using Unity Catalog for data access control, managing external storage and tables, data segregation, and more. Learning objectives Describe fundamental concepts about using Unity Catalog for data governanceDescribe data access patterns in Databricks. Define data access rules and manage data ownership Secure access to external storage Upgrade legacy data assets to Unity CatalogPrerequisites Familiarity with the Databricks Lakehouse completion (completion of the course Fundamentals of the Databricks Lakehouse Platform V2)Familiarity with data governance topicsLast course update April 2023