Skip to main content

Advanced Techniques with Apache Spark Declarative Pipelines - Mandarin Chinese

本课程探讨 Databricks 的 Apache Spark Declarative Pipelines(SDP),用于构建生产级流式处理管道。您将学习高级设计模式、严格的数据质量管控以及跨平台集成——这些都是现实世界湖仓工程不可或缺的能力。


在整个课程中,您将深入了解现代数据采集与处理技术,掌握 Liquid Clustering 等布局优化工具以及用于混合 schema 事件的多路复用流式处理模式。完成所有模块后,您将能够自信地处理 schema演进、自动化变更数据捕获(CDC)并确保数据完整性。


注意:

  1. 本课程是 ‘Advanced Data Engineering with Databricks’ 系列的第一门课程。
  2. 对于 SCORM 讲座文件,请确保在完成内容后关闭 SCORM 窗口。请勿点击‘Next Lesson’按钮,否则 SCORM 模块可能无法标记为已完成。
Skill Level
Professional
Duration
3h
Prerequisites

本课程的内容是为具备以下技能、知识和能力的学员开发的:  

• Spark Declarative Pipelines — 完成"Build Data Pipelines with Apache Spark Declarative Pipelines"课程,或熟悉 CREATE OR REFRESH STREAMING TABLE、CONSTRAINTS 以及 Pipelines UI

• Delta Lake Fundamentals — 了解 Delta 表,以及 Delta 如何管理数据文件和事务日志

• Streaming Concepts — 掌握 SDP 中的微批流式处理、检查点和事件时间处理相关知识

• SQL Proficiency — 能够读写 SQL,包括 SELECT、JOIN、MERGE、CASE WHEN 以及常用聚合函数

• Python in Databricks Notebooks — 能够在 Databricks Notebooks 中读取和运行 Python 代码

• Unity Catalog Basics — 了解 Unity Catalog 中的目录、架构、表和 Volumes

Self-Paced

Custom-fit learning paths for data, analytics, and AI roles and career paths through on-demand videos

See all our registration options

Registration options

Databricks has a delivery method for wherever you are on your learning journey

Runtime

Self-Paced

Custom-fit learning paths for data, analytics, and AI roles and career paths through on-demand videos

Register now

Instructors

Instructor-Led

Public and private courses taught by expert instructors across half-day to two-day courses

Register now

Learning

Blended Learning

Self-paced and weekly instructor-led sessions for every style of learner to optimize course completion and knowledge retention. Go to Subscriptions Catalog tab to purchase

Purchase now

Scale

Skills@Scale

Comprehensive training offering for large scale customers that includes learning elements for every style of learning. Inquire with your account executive for details

Upcoming Public Classes

Questions?

If you have any questions, please refer to our Frequently Asked Questions page.