Magnite, a global adtech leader, processes trillions of daily ad requests across video, display, and streaming platforms. As data volumes surged and systems grew fragmented, the company needed a unified platform for governance, performance, and interoperability. With Databricks and Unity Catalog, Magnite centralized its data architecture, adopted an Iceberg-first strategy, and significantly improved performance and cost efficiency at scale.
Fragmented systems and massive scale create operational complexity
Magnite’s data challenges were shaped by both scale and structure. The company processes more than two trillion ad requests each day, generating a continuous stream of data that must be ingested, transformed, and made available for analytics and client-facing use cases. At the same time, the business had evolved through multiple acquisitions, resulting in three distinct platforms with their own data environments, access models, and workflows.
This combination made it difficult to operate efficiently. Data lived in different systems, governance policies were inconsistent, and teams often had to work across disconnected environments to access what they needed. As data volumes continued to grow, these limitations became more pronounced, affecting both performance and the ability to scale operations.
“We were managing data across multiple environments without a consistent way to govern access or share data across teams,” said Satish Nekkalapudi, Chief Architect of Data Engineering. “At our scale, that complexity adds up quickly.”
Centralizing governance with Unity Catalog and open formats
To address these challenges, Magnite adopted Databricks and Unity Catalog to unify its data platform and governance model. From the beginning, Unity Catalog served as the central layer for managing permissions, access, and data across the organization. Instead of maintaining separate user models and policies across systems, Magnite adopted a single, consistent governance approach.
“Having one catalog with consistent permissions changed how we operate,” said Kayvon Raphael, Head of Data Engineering. “It reduces overhead and makes it much easier for teams to access and work with data across the company.”
At the same time, Magnite adopted an Iceberg-first strategy for its most important datasets, particularly those consumed by external systems and clients. This approach allows data to be accessed openly across engines while remaining governed through a single source of truth. Internally, teams benefit from optimized performance, while externally, clients can access data using the tools that best fit their workflows.
This combination of centralized governance and open data formats enables Magnite to support a wide range of use cases—from internal analytics and data science to external data sharing—without introducing additional complexity.
Driving performance and cost efficiency at scale
Magnite’s new architecture delivered its most significant impact when applied to one of its largest and most critical datasets: a 900-terabyte log-level dataset that ingests approximately 25 billion rows every hour. Previously, this dataset was difficult to manage efficiently, with slow ingestion, costly clustering operations, and queries that could take hours to complete.
After migrating to managed Iceberg on Databricks, performance and efficiency improved dramatically. Average query times dropped by 50-70 percent, while existing consumer queries continued to run without modification. Data loading costs decreased by 33 percent, and overall warehouse compute costs for consumers were reduced by 70 percent. These gains were driven in part by predictive optimization, which automatically handles clustering, compaction, and data layout in the background.
“Predictive optimization removes a lot of the manual work we used to manage ourselves,” said Adam Venci, Director of Engineering. “It allows us to improve performance while also reducing cost.”
The improvements were especially meaningful for smaller client-facing workloads that accessed data through direct sharing. Previously, those queries had to scan large amounts of unrelated data because smaller customer datasets were interleaved with the rest of the platform’s traffic. After the migration, some workloads scanned 98% less data, dramatically improving query performance and resolving a major pain point for shared data consumers.
Beyond performance, Magnite also modernized how it shares data with clients. By enabling direct, governed data sharing, the company eliminated the need for manual data transfers and reduced the risk of losing governance controls outside the platform. This allows advertisers, agencies, and publishers to securely exchange data while maintaining compliance and control.
With Databricks and Unity Catalog, Magnite has built a foundation that supports both its current scale and future growth. By centralizing governance and standardizing on open formats, the company can continue to expand its data platform without reintroducing the complexity it set out to eliminate.
