Model Development at Scale

In this course, you will learn to design and implement machine learning workflows that scale from a single machine to distributed compute on Databricks. The curriculum is organized into three modules: foundations of model development (when and why to distribute, and how Spark executes work), scalable model development with Spark ML and MLflow, and distributed model training with Ray and PyTorch — closing with a framework for choosing the right tool for a given workload.

Through hands-on demos and labs, you will build distributed Spark ML pipelines using Transformers, Estimators, and the fit/transform paradigm; scale existing pandas code with the Pandas API on Spark; and track, compare, and govern experiments with MLflow 3 and Unity Catalog. You will then train models across a cluster with Ray Train and PyTorch (TorchDistributor and DeepSpeed), and learn to match each workload to the right framework based on data size, model size, hardware, and cost.

The course concludes with a capstone that takes a problem end to end — engineering features, training a model at scale, tracking it with MLflow, and registering it in Unity Catalog with a governance alias. (Production deployment, serving, and monitoring are covered in the separate Advanced MLOps course.)

By the end, you will be able to choose, build, and justify distributed, reproducible, and governed model-development workflows on Databricks.

Note: For SCORM lecture files, please ensure that you close the SCORM window after completing the content. Do not click the ‘Next Lesson’ button, as doing so may prevent the SCORM module from being marked as complete.

Skill Level

Professional

Duration

Prerequisites

This course is best suited for learners with the following background:

• The user should have intermediate-level knowledge of traditional machine learning concepts, development, and the use of Python and Git for ML projects.

• It is recommended that the user has intermediate-level experience with Python.

• Databricks Academy content is designed to be run on the provided Vocareum environment only.

• Databricks does not guarantee that the content can be run successfully on any other instances of a Databricks Workspace.

Self-Paced

Custom-fit learning paths for data, analytics, and AI roles and career paths through on-demand videos

Customer registration Partner registration

See all our registration options

Registration options

Databricks has a delivery method for wherever you are on your learning journey

Self-Paced

Custom-fit learning paths for data, analytics, and AI roles and career paths through on-demand videos

Instructor-Led

Public and private courses taught by expert instructors across half-day to two-day courses

Blended Learning

Self-paced and weekly instructor-led sessions for every style of learner to optimize course completion and knowledge retention. Go to Subscriptions Catalog tab to purchase

Purchase now

Skills@Scale

Comprehensive training offering for large scale customers that includes learning elements for every style of learning. Inquire with your account executive for details

Upcoming Public Classes

AI Engineer

Get Started with AI Agents on Databricks - Mandarin Chinese

本课程介绍 AI 智能体的概念、其在现代 AI 应用中的作用，以及如何在 Databricks 平台上构建 AI 智能体应用程序。您将学习 AI 智能体的基本原理、它们与传统 AI 系统的区别，并浏览其核心键组件。通过互动演示和实践实验，您将了解如何使用 Databricks AI 和 Agent Bricks 在 Databricks 上构建、部署和评估 AI 智能体。在本课程结束时，您将掌握如何使用 Databricks Data + AI Platform 创建和部署 AI 智能体。

注意：Databricks Academy 正在将 Databricks 环境中的课堂课程转为基于 notebook 的形式，并停止在讲座中使用幻灯片。您可以在 Vocareum 实验环境中访问课程 notebook。

Get Started with AI Agents on Databricks

This course is an introduction to AI agents, their role in modern AI applications, and how to build AI agent applications on the Databricks platform. You'll learn about the principles of AI agents, how they differ from traditional AI systems, and explore their key components. Through interactive demos and hands-on labs, you'll see how to build, deploy, and evaluate AI agents on Databricks using Databricks AI and Agent Bricks. By the end of this course, you'll understand how to create and deploy AI agents using the Databricks Data + AI Platform.

Introduction to Lakebase

Establishes what Lakebase is and why it exists as fully managed Postgres on the lakehouse with unified governance. Built on the lecture-demo pattern (teach a concept, immediately run it), the course anchors four canonical use cases — each proven with a named customer story — and a three-signal decision framework for choosing between Lakebase, Delta, and the Lakehouse. It is the entry point for the Lakebase pathway; the use cases and decision tree introduced here are expanded by every later course in the series.

Paid & Subscription

Lab

Associate