Databricks Performance Optimization

Databricks Performance Optimization teaches data engineers and analysts how to diagnose, measure, and fix performance bottlenecks on the Databricks Data Intelligence Platform, and how to tie performance improvements back to cost. The course follows a measurement-first, platform-first workflow: let the platform's automatic optimizations do the heavy lifting, verify what was applied using Query Profile and system tables, then manually tune where needed.

Learners build a foundation for measuring performance with Query Profile, Performance Insights, and query history system tables. From there, they explore key optimization techniques, including data layout and self-tuning managed tables, liquid clustering, caching and intermediate results, shuffle, skew, spill, row explosions, driver performance, Python UDFs, serverless compute, Photon, and cost attribution.

Throughout the course, learners explore deliberately slow queries over synthetic retail data and see how different optimization techniques affect performance. They use evidence such as file pruning, task execution time, Photon coverage, and cost to evaluate improvements. Two scenario-based labs reinforce the optimization workflow from diagnosis through validation.

Note: For SCORM lecture files, please ensure that you close the SCORM window after completing the content. Do not click the ‘Next Lesson’ button, as doing so may prevent the SCORM module from being marked as complete.