Data Pipelines with Delta Live Tables
Learning objectives
- Describe how Delta Live Tables tracks data dependencies in data pipelines.
- Configure and run data pipelines using the Delta Live Tables UI.
- Use Python or Spark SQL to define data pipelines that ingest and process data through multiple tables in the lakehouse using Auto Loader and Delta Live Tables.
- Use APPLY CHANGES INTO syntax to process Change Data Capture feeds.
- Review event logs and data artifacts created by pipelines and troubleshoot DLT syntax
Prerequisites
- Beginner familiarity with cloud computing concepts (virtual machines, object storage, etc.)
- Ability to perform basic code development tasks using the Databricks Data Engineering & Data Science workspace (create clusters, run code in notebooks, use basic notebook operations, import repos from git, etc)
- Beginning programming experience with Delta Lake,Use Delta Lake DDL to create tables, compact files, restore previous table versions, and perform garbage collection of tables in the Lakehouse.Use CTAS to store data derived from a query in a Delta Lake table.Use SQL to perform complete and incremental updates to existing tables.
- Beginner programming experience with Python (syntax, conditions, loops, functions)
- Beginning programming experience with Spark SQL or PySpark.
- Extract data from a variety of file formats and data sources.
- Apply a number of common transformations to clean data.
- Reshape and manipulate complex data using advanced built-in functions.
- Production experience working with data warehouses and data lakes.
Last course update
- April 2023
Outline
- Introduction to the medallion architecture
- Introduction to Delta Live Tables (DLT)
- How to use DLT
- Exploring the DLT UI
- Python vs SQL with DLT
- Ingesting data
- Using parameters in DLT pipelines
- Enforcing data quality
- Configure DLT pipelines
Upcoming Public Classes
Date | Time | Language | Price |
---|---|---|---|
Jun 03 | 02 PM - 06 PM (America/New_York) | English | $750.00 |
Jun 06 | 02 PM - 06 PM (Europe/London) | English | $750.00 |
Jun 26 | 09 AM - 01 PM (Australia/Sydney) | English | $750.00 |
Jul 09 | 02 PM - 06 PM (Asia/Kolkata) | English | $750.00 |
Jul 10 | 09 AM - 01 PM (Europe/London) | English | $750.00 |
Jul 11 | 09 AM - 01 PM (America/New_York) | English | $750.00 |
Public Class Registration
If your company has purchased success credits or has a learning subscription, please fill out the Training Request form. Otherwise, you can register below.
Private Class Request
If your company is interested in private training, please submit a request.
Registration options
Databricks has a delivery method for wherever you are on your learning journey
Self-Paced
Custom-fit learning paths for data, analytics, and AI roles and career paths through on-demand videos
Register nowInstructor-Led
Public and private courses taught by expert instructors across half-day to two-day courses
Register nowBlended Learning
Self-paced and weekly instructor-led sessions for every style of learner to optimize course completion and knowledge retention. Go to Subscriptions Catalog tab to purchase
Purchase nowSkills@Scale
Comprehensive training offering for large scale customers that includes learning elements for every style of learning. Inquire with your account executive for details
Upcoming Public Classes
Career Workshop/
March 20