Apache Spark™ Programming with Databricks

Description
In this course, you will explore the fundamentals of Apache Spark and Delta Lake on Databricks. You will learn the architectural components of Spark, the DataFrame and Structured Streaming APIs, and how Delta Lake can improve your data pipelines. Lastly, you will execute streaming queries to process streaming data and understand the advantages of using Delta Lake.
This course will prepare you to take the Databricks Certified Associate Developer for Apache Spark exam.
Duration
2 full days or 4 half days
Objectives
Define Spark’s architectural components
Describe how DataFrames are transformed, executed, and optimized in Spark
Apply the DataFrame API to explore, preprocess, join, and ingest data in Spark
Apply the Structured Streaming API to perform analysis on streaming data
Use Delta Lake to improve the quality and performance of data pipelines
Prerequisites
Completion of Introduction to Python for Data Science & Data Engineering, OR familiarity with Python and basic programming concepts, including data types, lists, dictionaries, variables, functions, loops, conditional statements, exception handling, accessing classes, and using third-party libraries
Basic knowledge of SQL, including writing queries using
SELECT, WHERE, GROUP BY, ORDER BY, LIMIT, and JOIN
Outline
Day 1
Spark overview
Databricks platform overview
Spark SQL
DataFrame reader, writer, transformation, and aggregation
Datetimes
Complex types
Day 2
User-defined functions (UDFs) and vectorized UDFs
Spark internals
Query optimization
Partitioning
Streaming API
Delta Lake
Upcoming Public Classes
Public Class Registration
If your company has purchased success credits or has a learning subscription, please fill out the public training requests form. Otherwise, you can register below.
Private Class Delivery
If your organization would like to request a private delivery of the course, please fill out the request form below.
Questions?
If you have any questions, please refer to our Frequently Asked Questions page.