Building Semantic Models in Databricks with UC Metric Views

In this course, you will develop an in-depth understanding of how to design, implement, and govern scalable machine learning systems that operate effectively at enterprise scale. The curriculum is organized into three experiential modules: developing distributed ML workflows with frameworks such as Apache SparkML and Ray, transitioning local ML development to distributed compute using tools like Pandas on Spark, and operationalizing and governing production models with Databricks’ MLOps ecosystem.

Through hands-on projects, you will construct end-to-end distributed ML pipelines using the SparkML workflow, applying Transformers, Estimators, and the fit/transform paradigm for both classification and regression tasks. You will version, compare, and manage experiments using MLflow 3.0 to ensure reproducibility and governance, capturing lineage between data, features, and model artifacts. Additionally, you will apply scalable Hyperparameter Optimization frameworks to improve model performance at scale.

The course concludes by demonstrating complete lifecycle management, from experimentation to production deployment, using Unity Catalog and Model Serving. You will learn to operationalize trained models, monitor their performance, and implement strong governance over models, features, and Delta assets within the Databricks environment.

This course introduces Databricks Apps, a new feature that empowers developers to build and deploy secure data and AI applications directly on the Databricks platform. It is primarily designed for data scientists, data engineers, and developers who need to create custom applications that make data insights accessible to non-technical users within their organizations.The course begins with an overview of the main components of Databricks Apps, followed by the deployment of a simple app using available templates. Next, you will learn how to build and deploy a data-driven application that integrates with other platform features. The course then covers how to set up an IDE for Databricks Apps, and how to edit and deploy an app using an IDE. It concludes with how to implement security and best practices for an enterprise app.

Note: Databricks Academy is transitioning from video lectures to a more streamlined PDF format with slides and notes for all self-paced courses. Please note that demo videos will still be available in their original format. We would love to hear your thoughts on this change, so please share your feedback through the course survey at the end. Thank you for being a part of our learning community!

This course is meant to provide guidance on best practices for executing a successful migration from Enterprise Data Warehouses (EDWs) - e.g. Netezza, Teradata, Oracle, and Microsoft SQL Server - to Databricks. It covers a wide range of migration processes starting from initial discovery all the way through to delivery. 

Note: Databricks Academy is transitioning from video lectures to a more streamlined PDF format with slides and notes for all self-paced courses. Please note that demo videos will still be available in their original format. We would love to hear your thoughts on this change, so please share your feedback through the course survey at the end. Thank you for being a part of our learning community!