Stream Processing and Analysis with Apache Spark™

This course guides learners on how to develop scalable, fault-tolerant stream processing applications using the Spark Structured Streaming API on Databricks.

Note: For SCORM lecture files, please ensure that you close the SCORM window after completing the content. Do not click the ‘Next Lesson’ button, as doing so may prevent the SCORM module from being marked as complete.

Skill Level

Associate

Duration

Prerequisites

The content was developed for participants with these skills/knowledge/abilities:

• Basic programming knowledge

• Familiarity with Python

• Basic understanding of SQL queries (SELECT, JOIN, GROUP BY)

• Familiarity with data processing concepts

• Basic understanding of Apache Spark

Self-Paced

Custom-fit learning paths for data, analytics, and AI roles and career paths through on-demand videos

Customer registration Partner registration

See all our registration options

Registration options

Databricks has a delivery method for wherever you are on your learning journey

Self-Paced

Custom-fit learning paths for data, analytics, and AI roles and career paths through on-demand videos

Instructor-Led

Public and private courses taught by expert instructors across half-day to two-day courses

Blended Learning

Self-paced and weekly instructor-led sessions for every style of learner to optimize course completion and knowledge retention. Go to Subscriptions Catalog tab to purchase

Purchase now

Skills@Scale

Comprehensive training offering for large scale customers that includes learning elements for every style of learning. Inquire with your account executive for details

Upcoming Public Classes

Apache Spark Developer

Monitoring and Optimizing Apache Spark Workloads on Databricks

This course covers core Databricks platform components and Apache Spark with Delta Lake, focusing on the Lakehouse architecture, Unity Catalog, and data management techniques like ACID transactions and table maintenance. Additionally, it teaches students how to optimize query performance through partitioning, shuffles, and tuning tools, while utilizing the Spark UI to monitor applications and resolve bottlenecks.

Languages Available: English | 日本語 | 한국어

Paid & Subscription

Lab

Associate

Get Started with Databricks: End to End - Mandarin Chinese

本课程是对 Databricks Data Intelligence Platform 的全面实践入门课程。您将跟随一段完整的端到端学习旅程，从工作区出发，最终完成对 AI 与机器学习的入门介绍，涵盖在 Databricks 上处理数据所需的日常技能。

课程内容包括：

导航 Databricks 工作区、compute 选项，以及湖仓与 Genie 体验的对比。通过 Unity Catalog 理解和摄取数据，包括文件上传与 Catalog Explorer 的使用。在 Databricks Notebooks 中工作，并使用 Genie Code 编写和调试代码。使用 SQL 查询数据、管理已保存的查询，以及构建仪表盘和可视化图表。通过 Genie Agents 以自然语言探索数据。借助 MLflow 和 Data Science Agent 进行 AI 与机器学习入门。

完成本课程后，您将能够熟练导航 Databricks、在 Unity Catalog 中查找和管理数据、在 AI 辅助下编写 SQL 和笔记本代码、构建仪表盘，以及使用 Genie 探索数据。

注意：对于 SCORM 课程文件，请在完成内容后关闭 SCORM 窗口。请勿点击“Next Lesson”按钮，否则 SCORM 模块可能无法标记为已完成。

Languages Available: English | 日本語 | 한국어 | 普通话 | Português | Español | Français

External Apps with Lakebase: SDKs and Production Patterns

Connects apps running outside Databricks to Lakebase. Following the lecture-demo pattern, learners authenticate an external app with a service principal, compare the SDK-native, manual-HTTP, and Data API access paths, and build a production connection manager with pooling, token refresh, and retry on scale-to-zero wake-up.

Paid & Subscription

Lab

Associate