The Lakehouse Behind a Modern CDP

Building a CDP isn’t just a UI and APIs—it’s a data platform problem at scale. This talk shows how Warner Bros. Discovery built the Databricks-powered “back of house” for Mosaic, supporting audience segmentation and activation across millions of subscribers. We’ll walk through the data plane that converts high-volume first-party events (client SDK + server events) into governed Delta tables using Spark Structured Streaming, then runs incremental batch pipelines to produce custom attributes and curated event views for segmentation across identity grains. We’ll also cover the audience evaluation engine: drag-and-drop audience definitions are stored as rule trees, compiled into Spark plans and evaluated at scale to generate deterministic audience outputs for downstream delivery. Key production patterns include table modeling, join/partition strategies, observability and separation between the metadata plane (API-driven config) and the Databricks execution plane.