Product descriptions:
As one of the UK’s leading insurers, Allianz UK Personal Lines protects millions of customers across motor, home, travel and roadside assistance coverage, along with commercial and pet insurance through their broader business portfolio. To keep pace with growing data demands and accelerate decision-making in a competitive market, the company set out to modernize their data platform. Legacy on-premises warehouses and SAS processes were costly, complex and slow to scale. Since migrating to Databricks SQL, Allianz UK Personal Lines has streamlined operations, reduced costs and delivered data to business users in minutes rather than days, building a foundation for AI-ready analytics and smarter insurance decisions.
Legacy systems limited data speed and collaboration
Across their personal lines business, which encompasses car, home, travel and roadside assistance insurance, Allianz UK Personal Lines relied on a number of data systems that had grown over decades. Different teams maintained their own SQL Server, Oracle and SAS environments, each producing slightly different answers from the same data. As data volumes grew, batch loads were taking half a day to complete, far exceeding the morning reporting deadlines required for pricing and risk teams. “Seven years ago, our data was very siloed, which made governance challenging,” Paul McComish, Head of Data Engineering at Allianz UK Personal Lines, recalled. “There was no collaboration between teams and no consistent way to manage data quality.”
To address this, Allianz UK Personal Lines established their Business Intelligence Center of Excellence (BICOE), a central team responsible for developing and maintaining the company’s modern data platform. The goal was clear: consolidate data, improve speed and accuracy, and provide business users with trusted access to information they could act upon. Initial attempts to scale traditional data warehouses proved costly and slow. Allianz UK Personal Lines needed a modern platform that could separate compute from storage, handle large-scale processing and support real-time analytics without constant manual tuning.
From complexity to a scalable, unified platform
The migration to Databricks SQL transformed Allianz UK Personal Lines’ data operations. By consolidating data engineering and analytics on a single lakehouse architecture, the team reduced processing times by 60%, cutting overnight loads from lunchtime finishes to before 5 AM, and saved approximately £80,000 per month in compute costs.
Serverless compute lets the team scale automatically and allocate resources by workload, ensuring that dashboards and pipelines never compete for capacity. From an operations perspective, Databricks has simplified infrastructure management. “The ability to spin up separate compute resources for different workloads, such as SQL queries and ETL jobs, is game-changing,” Paul said. “Unlike legacy systems that required specialized tuning, Databricks SQL scales automatically and just works.”
Allianz UK Personal Lines is rewriting decades-old SAS logic into Databricks SQL and Python, improving governance and maintainability while eliminating opaque, black-box processes. Spark ™ Declarative Pipelines (SDP, formerly DLT) automates change-data-capture pipelines that continuously update operational data, and Unity Catalog governs access across personal lines. Because Databricks SQL runs on the same platform as AI and ML capabilities, Allianz UK Personal Lines can prepare and serve AI-ready data directly from their warehouse, eliminating data movement and ensuring governance across the entire analytics lifecycle.
Driving faster decisions and business impact
Today, Databricks SQL is the foundation for insurance operations at Allianz UK Personal Lines. Data that once took two days to arrive is now available within minutes, allowing pricing and risk teams to respond quickly to market changes and emerging trends. For example, Allianz UK Personal Lines utilizes streaming data pipelines constructed with Spark Declarative Pipelines to integrate policy and claims data from cloud-based systems into Databricks SQL, providing teams with a near real-time copy of the data. Analysts connect BI tools directly to Databricks SQL to build dashboards that track quote volumes, claims patterns and pricing performance at scale. With Databricks SQL processing tens of terabytes and thousands of daily queries, teams are gaining real-time visibility into performance across the business.
By separating compute for business intelligence and data engineering workloads, Allianz UK Personal Lines has eliminated performance bottlenecks and reduced the need for constant manual optimization. This flexibility not only improves speed and cost efficiency but also frees architects and analysts to focus on higher-value work. “With Databricks SQL, our teams spend less time managing infrastructure and more time delivering insights that make a real business impact,” Paul said.
With Databricks SQL as their foundation, Allianz UK Personal Lines has significantly progressed through their migration journey and plans to complete the transition next year, unlocking a fully modernized lakehouse architecture that enables real-time insights and future AI innovation across the organization.