Magnite's Journey of Interoperability on a Petabyte of Iceberg Data

For Magnite's business, they have a need to handle the ingest of 900TB of data every 10 days, while offering subsets of that data to many different customers (internal and external). Join this session to learn how Magnite successfully accomplished this while centralizing their data access via Unity Catalog while improving performance and handling many different engines (including Apache Spark, DuckDB, and Snowflake). We'll discuss the different approaches required, best practices followed, and rough edges encountered along the way. This lively presentation will be beneficial to anyone looking to separate compute and storage in their data domain at any scale.