Data Engineering with Databricks
Data professionals from all walks of life will benefit from this comprehensive introduction to the components of the Databricks Lakehouse Platform that directly support putting ETL pipelines into production. You will leverage SQL and Python to define and schedule pipelines that incrementally process new data from a variety of data sources to power analytic applications and dashboards in the Lakehouse. This course offers hands-on instruction in Databricks Data Science & Engineering Workspace, Databricks SQL, Delta Live Tables, Databricks Repos, Databricks Task Orchestration, and the Unity Catalog.
Languages Available: English | 日本語 | Português BR | 한국어
- Before attempting this course, please ensure that you meet these prerequisites. If you don’t, you might have trouble following along with course content.
Note that this course provides context for how to perform the actions within it using both Spark SQL and PySpark. If you are interested in one over the other, feel free to focus on that content (and skip the videos that are not relevant to you).
If you plan to follow along in Spark SQL, here are your pre-requisites:
- * Beginner familiarity with cloud computing concepts (virtual machines, object storage, etc.)
- * Production experience working with data warehouses and data lakes
- * Intermediate experience with basic SQL concepts (select, filter, groupby, join, etc)
If you plan on following along in both Spark SQL and PySpark, please see these additional pre-requisites:
* Beginner programming experience with Python (syntax, conditions, loops, functions)
* Beginner programming experience with the Spark DataFrame API (PySpark)
* Configure DataFrameReader and DataFrameWriter to read and write data (PySpark)
* Express query transformations using DataFrame methods and Column expressions (PySpark)
* Navigate the Spark documentation to identify built-in functions for various transformations and data types (PySpark)
Outline
Day 1:
Module 1: Get Started with Databricks Data Science and Data Engineering Workspace
Module 2: Transform Data with Spark
Module 3: Manage Data with Delta Lake
Day 2:
Module 4: Build Data Pipelines with Delta Live Tables
Module 5: Deploy Workloads with Databricks Workflows
Module 6: Manage Data Access with Unity Catalog
Upcoming Public Classes
Date | Time | Language | Price |
|---|---|---|---|
Sep 16 - 19 | 09 AM - 01 PM (America/Los_Angeles) | English | $1500.00 |
Sep 23 - 26 | 01 PM - 05 PM (America/Chicago) | Spanish | $1500.00 |
Sep 24 - 25 | 09 AM - 05 PM (Europe/Paris) | English | $1500.00 |
Sep 30 - Oct 01 | 09 AM - 05 PM (Europe/Paris) | English | $1500.00 |
Sep 30 - Oct 01 | 09 AM - 05 PM (Europe/London) | English | $1500.00 |
Oct 07 - 10 | 01 PM - 05 PM (Australia/Sydney) | English | $1500.00 |
Oct 07 - 08 | 09 AM - 05 PM (America/Los_Angeles) | English | $1500.00 |
Oct 14 - 15 | 09 AM - 05 PM (Europe/Paris) | English | $1500.00 |
Oct 16 - 17 | 09 AM - 05 PM (Europe/London) | English | $1500.00 |
Oct 21 - 24 | 01 PM - 05 PM (America/New_York) | English | $1500.00 |
Oct 22 - 25 | 09 AM - 01 PM (Asia/Kolkata) | English | $1500.00 |
Oct 22 - 25 | 01 PM - 05 PM (America/New_York) | English | $1500.00 |
Oct 24 - 25 | 09 AM - 05 PM (Europe/Paris) | English | $1500.00 |
Oct 28 - 29 | 09 AM - 05 PM (Europe/London) | English | $1500.00 |
Nov 05 - 08 | 01 PM - 05 PM (Australia/Sydney) | English | $1500.00 |
Nov 07 - 08 | 09 AM - 05 PM (America/New_York) | English | $1500.00 |
Nov 18 - 21 | 10 AM - 02 PM (Asia/Singapore) | English | $1500.00 |
Nov 18 - 19 | 09 AM - 05 PM (Europe/Paris) | English | $1500.00 |
Nov 18 - 21 | 09 AM - 01 PM (America/Denver) | English | $1500.00 |
Nov 25 - 26 | 09 AM - 05 PM (Europe/London) | English | $1500.00 |
Dec 02 - 03 | 09 AM - 05 PM (Europe/Paris) | English | $1500.00 |
Dec 03 - 06 | 01 PM - 05 PM (Australia/Sydney) | English | $1500.00 |
Dec 03 - 04 | 10 AM - 06 PM (America/Los_Angeles) | English | $1500.00 |
Dec 16 - 19 | 09 AM - 01 PM (Asia/Kolkata) | English | $1500.00 |
Dec 16 - 17 | 09 AM - 05 PM (Europe/Paris) | English | $1500.00 |
Dec 16 - 19 | 01 PM - 05 PM (America/New_York) | English | $1500.00 |
Jan 06 - 07 | 09 AM - 05 PM (Europe/London) | English | $1500.00 |
Jan 07 - 10 | 01 PM - 05 PM (Australia/Sydney) | English | $1500.00 |
Jan 13 - 16 | 01 PM - 05 PM (America/Chicago) | English | $1500.00 |
Jan 20 - 23 | 10 AM - 02 PM (Asia/Singapore) | English | $1500.00 |
Jan 27 - 28 | 09 AM - 05 PM (Europe/London) | English | $1500.00 |
Jan 27 - 28 | 09 AM - 05 PM (America/Los_Angeles) | English | $1500.00 |
Public Class Registration
If your company has purchased success credits or has a learning subscription, please fill out the Training Request form. Otherwise, you can register below.
Private Class Request
If your company is interested in private training, please submit a request.
Registration options
Databricks has a delivery method for wherever you are on your learning journey
Self-Paced
Custom-fit learning paths for data, analytics, and AI roles and career paths through on-demand videos
Register now
Instructor-Led
Public and private courses taught by expert instructors across half-day to two-day courses
Register now
Blended Learning
Self-paced and weekly instructor-led sessions for every style of learner to optimize course completion and knowledge retention. Go to Subscriptions Catalog tab to purchase
Purchase now
Skills@Scale
Comprehensive training offering for large scale customers that includes learning elements for every style of learning. Inquire with your account executive for details
Upcoming Public Classes
Career Workshop/
March 20
