Get Started with Databricks Platform Administration

Databricks Performance Optimization teaches data engineers and analysts how to diagnose, measure, and fix performance bottlenecks on the Databricks Data Intelligence Platform, and how to tie performance improvements back to cost. The course follows a measurement-first, platform-first workflow: let the platform's automatic optimizations do the heavy lifting, verify what was applied using Query Profile and system tables, then manually tune where needed.

Learners build a foundation for measuring performance with Query Profile, Performance Insights, and query history system tables. From there, they explore key optimization techniques, including data layout and self-tuning managed tables, liquid clustering, caching and intermediate results, shuffle, skew, spill, row explosions, driver performance, Python UDFs, serverless compute, Photon, and cost attribution.

Throughout the course, learners explore deliberately slow queries over synthetic retail data and see how different optimization techniques affect performance. They use evidence such as file pruning, task execution time, Photon coverage, and cost to evaluate improvements. Two scenario-based labs reinforce the optimization workflow from diagnosis through validation.

Note: For SCORM lecture files, please ensure that you close the SCORM window after completing the content. Do not click the ‘Next Lesson’ button, as doing so may prevent the SCORM module from being marked as complete.

This course covers what an administrator of a Databricks account and its workspaces needs to know, cloud-agnostic: platform architecture and admin roles, compute resource administration, identity administration, CI/CD tooling administration, and workspace administration and security. Cloud-specific mechanics (IAM, storage, networking) are covered in the companion cloud admin courses. Prepares learners for the Databricks Accredited Platform Administrator accreditation.

Note: For SCORM lecture files, please ensure that you close the SCORM window after completing the content. Do not click the ‘Next Lesson’ button, as doing so may prevent the SCORM module from being marked as complete.

在本课程中，您将学习如何设计和实现机器学习工作流，使其能够从单台机器扩展到 Databricks 上的分布式 compute。课程内容分为三个模块：模型开发基础（何时以及为何进行分布式处理，以及 Spark 如何执行任务）、使用 Spark ML 和 MLflow 进行可扩展模型开发，以及使用 Ray 和 PyTorch 进行分布式模型训练——最后介绍一个用于为特定工作负载选择合适工具的框架。

通过动手演示和实验，您将使用 Transformers、Estimators 以及 fit/transform 范式构建分布式 Spark ML管道；使用 Spark 上的 Pandas API 扩展现有 Pandas 代码；并使用 MLflow 3 和 Unity Catalog 追踪、比较和管理实验。随后，您将使用 Ray Train 和 PyTorch（TorchDistributor 和 DeepSpeed）在集群中训练模型，并学习如何根据数据规模、模型规模、硬件和成本，将每种工作负载与合适的框架相匹配。

课程最后以一个综合实践项目收尾，该项目涵盖端到端的完整流程——特征工程、大规模训练模型、使用 MLflow 进行追踪，以及通过治理别名将模型注册到 Unity Catalog。（生产部署、服务和监控内容将在独立的 Advanced MLOps 课程中讲授。）

学完本课程后，您将能够在 Databricks 上选择、构建并论证分布式、可复现且受治理的模型开发工作流。

注意：对于 SCORM 讲座文件，请在完成内容后关闭 SCORM 窗口。请勿点击“Next Lesson”按钮，否则可能导致 SCORM 模块无法标记为已完成。