Available on-demand!

Unified Data Analytics Webinar - Simplifying Streaming Analytics with Delta Lake and Spark

Simplifying Streaming Analytics with Delta Lake and Spark

Available on-demand

Given the rise of IoT and other real-time sources and businesses’ desire to draw fast insights, there is a growing imperative for data professionals to build streaming data pipelines. Given the plethora of different tools and frameworks in the big data community, it is challenging to architect such pipelines correctly that achieve the desired performance and data quality. Hence, it is important to properly understand the business and technical requirements of the pipelines and accordingly select the right tools to build them.

By combining the scalability, streaming, and access to advanced analytics of Apache Spark with the performance and ACID compliance of a data warehouse, Delta Lake helps solve many of the pain points of building a streaming system to analyze data in real-time.

In this webinar, we are going to show how Structured Streaming and Delta Lake together make it super-easy to write end-to-end pipelines. Specifically, we will cover the following:

  • How to approach the problem of designing a streaming pipeline
  • How to classify the pipelines into common design patterns
  • A demo on how to implement a pipeline using Spark Structured Streaming and Delta Lake on Databricks

Featured Speaker:
Sreeram Nudurupati, Senior Solutions Architect, Databricks