Catalog Management and Data Organization

In this course, you will learn how to design, implement, and govern catalog structures and large-scale data organization with Unity Catalog on the Databricks Data and AI Platform. Divided into five modules, it begins by placing Unity Catalog within the cloud deployment model, covering the account console, regional metastores, and the administrator roles that enable it. You will then translate organizational topology (business units, regions, and dev/QA/prod environments) into a scalable catalog and schema design with naming conventions, ownership, and MANAGE delegation. Next, the course covers secure storage integration with storage credentials, external locations, the managed-storage hierarchy, managed versus external tables, volumes, and Lakehouse Federation. You will then apply access patterns and isolation strategies, including workspace-catalog binding, privilege inheritance, row filters, column masks, and attribute-based access control (ABAC) policies, and validate the outcomes with audit signals. Finally, the course closes with best practices for designing, governing, and operating a Unity Catalog estate. The hands-on demos and labs run on serverless compute and include short Genie Code steps.

Note: For SCORM lecture files, please ensure that you close the SCORM window after completing the content. Do not click the ‘Next Lesson’ button, as doing so may prevent the SCORM module from being marked as complete.

Skill Level

Associate

Duration

Prerequisites

In this course, the content was developed for participants with these skills/knowledge/abilities:

• Familiarity with Databricks workspaces and SQL warehouses, including working with notebooks in SQL, Python, or Markdown

• A solid understanding of Unity Catalog and Delta Lake for organizing and managing data assets within the lakehouse

• Introductory awareness of Lakeflow for orchestrating and managing data workflows

• Experience in architecture within larger organizations, to understand the complexities of enterprise-level IT landscapes

• Practical experience querying and analyzing datasets using SQL, including exposure to parameterized queries and data migration tasks

• Hands-on experience with Unity Catalog-enabled workspaces, such as creating and managing catalogs, schemas, and tables

• A solid understanding of enterprise architecture frameworks, including capability mapping and logical architecture

• Working knowledge of data governance principles, including security, compliance, access control, and BI reporting concepts

• Competency in ANSI SQL and a conceptual understanding of how data pipelines and workflow orchestration work

Self-Paced

Custom-fit learning paths for data, analytics, and AI roles and career paths through on-demand videos

Customer registration Partner registration

See all our registration options

Registration options

Databricks has a delivery method for wherever you are on your learning journey

Self-Paced

Custom-fit learning paths for data, analytics, and AI roles and career paths through on-demand videos

Instructor-Led

Public and private courses taught by expert instructors across half-day to two-day courses

Blended Learning

Self-paced and weekly instructor-led sessions for every style of learner to optimize course completion and knowledge retention. Go to Subscriptions Catalog tab to purchase

Purchase now

Skills@Scale

Comprehensive training offering for large scale customers that includes learning elements for every style of learning. Inquire with your account executive for details

Upcoming Public Classes

Catalog Management and Data Organization - Mandarin Chinese

本课程将介绍如何在 Databricks Data and AI Platform 上使用 Unity Catalog 设计、实施和管理目录结构及大规模数据组织。课程共分五个模块：首先将 Unity Catalog 置于云平台部署模型的背景下，涵盖账户控制台、区域元存储以及实现其运作的管理员角色；随后，您将学习如何把组织拓扑结构（业务单元、地区及开发/QA/生产环境）转化为可扩展的目录和 schema 设计，并制定命名规范、所有权及 MANAGE 委派策略；接下来，课程将介绍安全存储集成，包括存储凭证、外部位置、托管存储层级、托管表与外部表、Volumes 以及 Lakehouse Federation；之后，您将应用访问模式和隔离策略，包括 workspace-目录绑定、权限继承、行过滤器、列掩码以及基于属性的访问控制（ABAC）策略，并通过审计信号验证结果；最后，课程以设计、管理和运营 Unity Catalog 资产的最佳实践作为总结。实操演示和实验在 serverless compute 上运行，并包含简短的 Genie Code 步骤。

注：SCORM講義ファイルの場合、コンテンツを完了したらSCORMウィンドウを閉じてください。「次のレッスン」ボタンはクリックしないでください。クリックすると、SCORMモジュールが完了として記録されない場合があります。

Advanced Techniques with Apache Spark Declarative Pipelines - Mandarin Chinese

本课程探讨 Databricks 的 Apache Spark Declarative Pipelines（SDP），用于构建生产级流式处理管道。您将学习高级设计模式、严格的数据质量管控以及跨平台集成——这些都是现实世界湖仓工程不可或缺的能力。

在整个课程中，您将深入了解现代数据采集与处理技术，掌握 Liquid Clustering 等布局优化工具以及用于混合 schema 事件的多路复用流式处理模式。完成所有模块后，您将能够自信地处理 schema演进、自动化变更数据捕获（CDC）并确保数据完整性。

注意：

本课程是 ‘Advanced Data Engineering with Databricks’ 系列的第一门课程。对于 SCORM 讲座文件，请确保在完成内容后关闭 SCORM 窗口。请勿点击‘Next Lesson’按钮，否则 SCORM 模块可能无法标记为已完成。

Paid & Subscription

Lab

Professional

Machine Learning Practitioner

Data Preparation for Machine Learning - Mandarin Chinese

本课程讲授在 Databricks 上为传统机器学习准备数据的基础知识。学员将数据加载到由 Unity Catalog 管理的 Delta 表中，并通过 Notebooks、内置可视化功能和 Genie Code 对数据进行浏览。随后，他们对数据进行数据清洗、处理缺失值和异常值、在不产生数据泄漏的前提下对数据进行拆分和采样，并构建可复现的 Spark ML管道，用于填补缺失值、编码和标准化。最后，他们使用 Unity Catalog 中的 Feature Engineering 对特征进行存储、管理和复用。内容还涵盖用于无泄漏训练集的时间点查询，以及基于 Lakebase 的 Online Feature Store 在线特征服务简介。每个模块均将讲授与实操演示相结合，并配有一个在 serverless compute 上运行的实验室。模型训练、服务和监控不在本课程范围内，相关内容在 Machine Learning Model Development 和 Machine Learning Model Deployment 课程中讲授。本课程面向希望在 Databricks 上夯实数据准备基础的初级数据科学家和机器学习从业者。

注意：

这是 'Machine Learning with Databricks’ 系列课程中的第一门课程。对于 SCORM 课程文件，请确保完成内容后关闭 SCORM 窗口。请勿点击 ‘Next Lesson’ 按钮，否则可能导致 SCORM 模块无法标记为已完成。

Paid & Subscription

Lab

Associate