Skip to main content

Apache Spark™ Programming with Databricks

In this course, you will explore the fundamentals of Apache Spark and Delta Lake on Databricks. You will learn the architectural components of Spark, the DataFrame and Structured Streaming APIs, and how Delta Lake can improve your data pipelines. Lastly, you will execute streaming queries to process streaming data and understand the advantages of using Delta Lake.

   
Skill Level
Associate
Duration
12h
Prerequisites
  • Familiarity with Python and basic programming concepts, including data types, lists, dictionaries, variables, functions, loops, conditional statements, exception handling, accessing classes, and using third-party libraries
  • Basic knowledge of SQL, including writing queries using SELECT, WHERE, GROUP BY, ORDER BY, LIMIT, and JOIN

Outline

Day 1

  • Spark overview
  • Databricks platform overview
  • SparkSQL
  • DataFrame reader, writer, transformation, and aggregation
  • Datetimes
  • Complex types


Day 2

  • User-defined functions (UDFs) and vectorized UDFs
  • Spark internals
  • Query optimization
  • Partitioning
  • Streaming API
  • Delta Lake

Upcoming Public Classes

Date
Time
Language
Price
Apr 22
01 PM - 05 PM (Australia/Sydney)
English
$1500.00
Apr 29
09 AM - 01 PM (America/New_York)
English
$1500.00
May 06
08 AM - 12 PM (Asia/Kolkata)
English
$1500.00
May 07
09 AM - 05 PM (Europe/Paris)
English
$1500.00
May 13
09 AM - 05 PM (Asia/Tokyo)
Japanese
$1500.00
May 13
02 PM - 06 PM (America/New_York)
English
$1500.00
May 21
09 AM - 05 PM (Europe/Paris)
English
$1500.00
May 28
09 AM - 01 PM (America/New_York)
English
$1500.00
Jun 03
09 AM - 05 PM (Australia/Sydney)
English
$1500.00
Jun 06
09 AM - 05 PM (Asia/Tokyo)
Japanese
$1500.00
Jun 10
02 PM - 06 PM (America/New_York)
English
$1500.00
Jun 18
09 AM - 01 PM (America/New_York)
English
$1500.00
Jun 24
09 AM - 05 PM (Australia/Sydney)
English
$1500.00
Jun 24
09 AM - 05 PM (Europe/London)
English
$1500.00
Jul 17
09 AM - 05 PM (Europe/Paris)
English
$1500.00

Public Class Registration

If your company has purchased success credits or has a learning subscription, please fill out the Training Request form. Otherwise, you can register below.

Private Class Request

If your company is interested in private training, please submit a request.

See all our registration options

Registration options

Databricks has a delivery method for wherever you are on your learning journey

Runtime

Self-Paced

Custom-fit learning paths for data, analytics, and AI roles and career paths through on-demand videos

Register now

Instructors

Instructor-Led

Public and private courses taught by expert instructors across half-day to two-day courses

Register now

Learning

Blended Learning

Self-paced and weekly instructor-led sessions for every style of learner to optimize course completion and knowledge retention. Go to Subscriptions Catalog tab to purchase

Purchase now

Scale

Skills@Scale

Comprehensive training offering for large scale customers that includes learning elements for every style of learning. Inquire with your account executive for details

Upcoming Public Classes

Data Engineer

Data Workloads with Repos and Workflows

Moving a data pipeline to production means more than just confirming that code and data are working as expected. By scheduling tasks with Databricks Jobs, applications can be run automatically to keep tables in the Lakehouse fresh. Using Databricks SQL to schedule updates to queries and dashboards allows quick insights using the newest data. In this course, students will be introduced to task orchestration using the Databricks Workflow Jobs UI. Optionally, they will configure and schedule dashboards and alerts to reflect updates to production data pipelines. Learning objectives Version code with Databricks ReposOrchestrate tasks with Databricks Workflow Jobs. Use Databricks SQL for on-demand queries. Configure and schedule dashboards and alerts to reflect updates to production data pipelines.Prerequisites Ability to perform basic code development tasks using the Databricks Data Engineering & Data Science workspace (create clusters, run code in notebooks, use basic notebook operations, import repos from git, etc) Ability to configure and run data pipelines using the Delta Live Tables UI. Beginner experience defining Delta Live Tables (DLT) pipelines using PySpark Ingest and process data using Auto Loader and PySpark syntax. Process Change Data Capture feeds with APPLY CHANGES INTO syntax Review pipeline event logs and results to troubleshoot DLT syntax Reshape and manipulate complex data using advanced built-in functions. Production experience working with data warehouses and data lakes. Last course update April 2023
Paid
4h
Lab
instructor-led
Associate
Career Workshop

Career Workshop/

March 20

Careers at Databricks

We're on a mission to help data teams solve the world's toughest problems. Will you join us?
Advance my career now

Questions?

If you have any questions, please refer to our Frequently Asked Questions page.