Machine Learning Model Development

This comprehensive course provides a practical guide to developing traditional machine learning models on Databricks, emphasizing hands-on demonstrations and workflows using popular ML libraries. Participants will explore key ML techniques, including regression and clustering, while leveraging Databricks’ powerful capabilities. The course covers MLflow integration for model tracking, Databricks Feature Store for feature management, and Optuna for hyperparameter tuning. Additionally, participants will learn how to accelerate model training with Databricks AutoML. By the end of the course, learners will have real-world, practical skills to develop, optimize, and deploy machine learning models efficiently in the Databricks environment.

Note:

This is the second course in the 'Machine Learning with Databricks’ series.
Databricks Academy is transitioning from video lectures to a more streamlined PDF format with slides and notes for all self-paced courses. Please note that demo videos will still be available in their original format. We would love to hear your thoughts on this change, so please share your feedback through the course survey at the end. Thank you for being a part of our learning community!

Skill Level

Associate

Duration

Prerequisites

At a minimum, you should be familiar with the following before attempting to take this content:

• Familiarity with the Databricks Data Intelligence Platform and basic workspace operations (create clusters, run code in notebooks, use basic notebook operations, import repos from git)

• Intermediate programming experience with Python, including data manipulation libraries (pandas, numpy) and working with APIs (databricks-sdk, REST endpoints)

• Basic knowledge of MLflow for experiment tracking, model logging, model registry operations, and model versioning

• Understanding of machine learning fundamentals, including model training, evaluation, batch inference, and real-time deployment concepts

• Intermediate experience with Unity Catalog for data governance and model registry management

• Basic familiarity with Feature Engineering concepts, including feature tables, feature lookups, and offline vs online feature stores

• Understanding of Delta Lake operations (create tables, perform updates, optimize files, and liquid clustering) and data storage optimization techniques

• Basic knowledge of Apache Spark and PySpark for distributed data processing and User Defined Functions (UDFs)

Self-Paced

Custom-fit learning paths for data, analytics, and AI roles and career paths through on-demand videos

Customer registration Partner registration

See all our registration options

Registration options

Databricks has a delivery method for wherever you are on your learning journey

Self-Paced

Custom-fit learning paths for data, analytics, and AI roles and career paths through on-demand videos

Instructor-Led

Public and private courses taught by expert instructors across half-day to two-day courses

Blended Learning

Self-paced and weekly instructor-led sessions for every style of learner to optimize course completion and knowledge retention. Go to Subscriptions Catalog tab to purchase

Purchase now

Skills@Scale

Comprehensive training offering for large scale customers that includes learning elements for every style of learning. Inquire with your account executive for details

Upcoming Public Classes

Model Development at Scale

In this course, you will develop an in-depth understanding of how to design, implement, and govern scalable machine learning systems that operate effectively at enterprise scale. The curriculum is organized into three experiential modules: developing distributed ML workflows with frameworks such as Apache SparkML and Ray, transitioning local ML development to distributed compute using tools like Pandas on Spark, and operationalizing and governing production models with Databricks’ MLOps ecosystem.

Through hands-on projects, you will construct end-to-end distributed ML pipelines using the SparkML workflow, applying Transformers, Estimators, and the fit/transform paradigm for both classification and regression tasks. You will version, compare, and manage experiments using MLflow 3.0 to ensure reproducibility and governance, capturing lineage between data, features, and model artifacts. Additionally, you will apply scalable Hyperparameter Optimization frameworks to improve model performance at scale.

The course concludes by demonstrating complete lifecycle management, from experimentation to production deployment, using Unity Catalog and Model Serving. You will learn to operationalize trained models, monitor their performance, and implement strong governance over models, features, and Delta assets within the Databricks environment.

Paid & Subscription

Lab

Professional

Data Warehousing Practitioner

Delivery Specialization: EDW Migration Best Practices

This course is meant to provide guidance on best practices for executing a successful migration from Enterprise Data Warehouses (EDWs) - e.g. Netezza, Teradata, Oracle, and Microsoft SQL Server - to Databricks. It covers a wide range of migration processes starting from initial discovery all the way through to delivery.

Note: Databricks Academy is transitioning from video lectures to a more streamlined PDF format with slides and notes for all self-paced courses. Please note that demo videos will still be available in their original format. We would love to hear your thoughts on this change, so please share your feedback through the course survey at the end. Thank you for being a part of our learning community!

Data Warehousing Practitioner

Delivery Specialization: CDW Migration Best Practices

This course is meant to provide guidance on best practices for executing a successful migration from Cloud Data Warehouses (CDWs) - e.g. Amazon Redshift, Snowflake, and Synapse Analytics - to Databricks. It covers a wide range of migration processes starting from initial discovery all the way through to delivery.