Simplifying home improvements with targeted customer engagement
hipages leverages Databricks to improve decision-making and customer support
Uptime for key report (Job Estimator) on Databricks compared with Redash
Projected annual infrastructure cost savings
hipages has revolutionized Australia’s home improvement industry by connecting over 4 million homeowners with tradies (tradespeople). Fueled by a robust data-driven culture, the company faced significant challenges with legacy data systems that hindered productivity and insight, requiring days to transition data between multiple warehouses. Self-service analytics was an impossibility. Partnering with Databricks, hipages overcame these hurdles by transforming their data architecture into a single lakehouse architecture, enhancing both speed and reliability. This shift was supported by features like Databricks SQL, which significantly accelerated data analysis and reporting, granting data analysts the autonomy to create dynamic dashboards and conduct rapid analyses. Now, with 100% uptime on key reports and projected annual cost savings of thousands of dollars, hipages’ data transformation capabilities are streamlined — opening new avenues for targeted customer engagement and data-driven decision-making.
Separate data warehouses stifle productivity and insight in home improvement
hipages stands as Australia’s largest online tradie marketplace, serving as a pivotal connection between homeowners looking to improve their homes and tradies seeking job opportunities. Founded over 20 years ago, hipages has facilitated over 4 million connections between Australian homeowners and more than 32,000 tradies, spanning a wide range of services from plumbing to painting. As Minas Kamel, Senior Product Manager at hipages, highlighted, “hipages simplifies home improvements by connecting homeowners who need to renovate or fix something with tradies who are looking for jobs.”
Driven by a robust, data-oriented culture, hipages harnesses extensive historical data to empower every operational facet. The data team, though modest in size — comprising nine analysts, four data engineers, an analytics engineer and a machine learning engineer — plays a critical role in maintaining hipages as a data-driven entity. “Every product decision is supported by data analysis, and all our business metrics are data-driven,” Minas noted, underscoring the integral use of machine learning in optimizing lead pricing for tradies.
Despite their advancements, hipages faced significant challenges due to legacy systems and a fragmented data management approach, which stifled productivity and hindered insightful analytics. “We ended up with three generations of data warehouses, ” Minas explained, detailing the transition from MySQL to Redshift and then to Athena. “When things broke, it took a while to figure out where the issue was.” This layered legacy system led to inefficiencies, with data taking two days on average to transition between warehouses — delaying access for analysts and decision-makers.
The lack of a unified platform meant that data analysts were dependent on data engineers to access and interpret data, making self-service analytics nearly impossible. “How would we have autonomous teams that have access to data with that convoluted platform architecture?” Minas questioned, illustrating the critical need for a more streamlined solution.
These compounded platform, productivity and insight challenges necessitated a shift toward a more modern, efficient data management solution. So, hipages chose the Databricks Data Intelligence Platform. This partnership aimed to consolidate their data architecture into a single lakehouse architecture, optimizing both the speed and reliability of data processes and enabling a true self-service analytics environment across the company.
Migrating to Databricks to achieve self-service
The Databricks Data Intelligence Platform offered a unified, streamlined solution that significantly improved efficiency, reliability and scalability. Minas described the impact of this transition, stating, “We transformed our stack from three data warehouses into one data lakehouse, using Databricks, which majorly streamlined our operations and allowed our data users more autonomy.”
Databricks SQL significantly accelerates the process of data analysis and reporting at hipages. Data analysts now have the capability to create dynamic dashboards and perform rapid analyses for running experiments. Minas noted the advantages, saying, “Databricks SQL, especially, has been a major win for us. Our data analysts prefer to use Databricks SQL for easy and quick analysis.”
The integration of Kafka for capturing CDC (change data capture) events from their production database and Fivetran for batch ingestion from various sources has streamlined the data flow into their systems. This setup supports real-time data processing and enhances the overall responsiveness of their data services.
dbt Labs plays a crucial role in hipages’ data platform strategy. This tool allows users with different levels of tech proficiency to create reusable and modular SQL code, simplifying complex data transformations. “By using dbt, we were able to standardize and modularize our data models, and automate the creation and testing of these models. For example, we have a single customer view model that is built in dbt and used by Braze for improving our marketing campaigns.” The use of Hightouch for reverse ETL (extract, transform, load) processes ensures that marketing platforms like Braze are seamlessly integrated with up-to-date data — enabling targeted customer interactions based on the latest insights.
Accelerated customer analytics and real-time reporting with 100% uptime
This migration from a multi-warehouse environment to Databricks not only optimized the performance through serverless SQL but also significantly reduced management overhead by automating the scaling of resources. Minas highlighted a specific improvement: “A good example of high impact is a report we used to have in Redash called Job Estimator, which our sales team uses to estimate which package a tradie should sign up to. Since we moved this report to Databricks five months ago, we have had zero incidents.”
Due to the success of Databricks’ infrastructure, hipages is set to finalize their migration plans by decommissioning all legacy systems and transitioning all existing cron jobs to dbt models. “We are expecting to save over thousands of dollars annually,” Minas noted.
Looking ahead, hipages plans to further enhance their self-service capabilities by implementing Databricks Genie. This component is designed to enable even nontech users to ask data-related questions and receive answers, further democratizing data access across the organization. Minas’ team is also focusing on enhancing their data governance practices with Unity Catalog, ensuring that data management remains robust and compliant with evolving industry standards.
As hipages continues to innovate and expand their data capabilities, the full integration of Databricks’ solutions promises not only to resolve existing challenges but also to pave the way for new opportunities in data-driven decision-making and customer engagement. This ongoing transformation underscores hipages’ commitment to maintaining their leadership in the home improvement industry through technological excellence and strategic foresight.
