Advanced Techniques with Apache Spark Declarative Pipelines - Mandarin Chinese
本课程探讨 Databricks 的 Apache Spark Declarative Pipelines(SDP),用于构建生产级流式处理管道。您将学习高级设计模式、严格的数据质量管控以及跨平台集成——这些都是现实世界湖仓工程不可或缺的 能力。
在整个课程中,您将深入了解现代数据采集与处理技术,掌握 Liquid Clustering 等布局优化工具以及用于混合 schema 事件的多路复用流式处理模式。完成所有模块后,您将能够自信地处理 schema演进、自动化变更数据捕获(CDC)并确保数据完整性。
注意:
- 本课程是 ‘Advanced Data Engineering with Databricks’ 系列的第一门课程。
- 对于 SCORM 讲座文件,请确保在完成内容后关闭 SCORM 窗口。请勿点击‘Next Lesson’按钮,否则 SCORM 模块可能无法标记为已完成。
本课程的内容是为具备以下技能、知识和能力的学员开发的:
• Spark Declarative Pipelines — 完成"Build Data Pipelines with Apache Spark Declarative Pipelines"课程,或熟悉 CREATE OR REFRESH STREAMING TABLE、CONSTRAINTS 以及 Pipelines UI
• Delta Lake Fundamentals — 了解 Delta 表,以及 Delta 如何管理数据文件和事务日志
• Streaming Concepts — 掌握 SDP 中的微批流式处理、检查点和事件时间处理相关知识
• SQL Proficiency — 能够读写 SQL,包括 SELECT、JOIN、MERGE、CASE WHEN 以及常用聚合函数
• Python in Databricks Notebooks — 能够在 Databricks Notebooks 中读取和运行 Python 代码
• Unity Catalog Basics — 了解 Unity Catalog 中的目录、 架构、表和 Volumes
Self-Paced
Custom-fit learning paths for data, analytics, and AI roles and career paths through on-demand videos
Registration options
Databricks has a delivery method for wherever you are on your learning journey
Self-Paced
Custom-fit learning paths for data, analytics, and AI roles and career paths through on-demand videos
Register nowInstructor-Led
Public and private courses taught by expert instructors across half-day to two-day courses
Register nowBlended Learning
Self-paced and weekly instructor-led sessions for every style of learner to optimize course completion and knowledge retention. Go to Subscriptions Catalog tab to purchase
Purchase nowSkills@Scale
Comprehensive training offering for large scale customers that includes learning elements for every style of learning. Inquire with your account executive for details

