Transforming the Australian rail industry’s digital ecosystem
Faster time to insight, optimizing freight logistics
Reduction in data engineering hours, decreasing overall costs
“Databricks Data Intelligence Platform empowers us to pioneer the digital transformation of Australia’s rail industry, and build a strong data foundation that has improved the resilience, availability and efficiency of our services to deliver goods in a safe and efficient manner.”
As the backbone of Australia’s supply chain, the Australia Rail Track Corporation (ARTC) plays a vital role in the management and monitoring of goods transportation across 8,500km of its rail network throughout Australia. ARTC collects vast amounts of data to monitor operations and ensure the timely delivery of cargo freight on its rail network. As part of its Digital Integration Strategy, ARTC endeavored to remove the complexity of its siloed data across several segregated data sources with a more modern, streamlined approach, in order to leverage data-driven insights and strategies — resulting in cost savings and increased efficiencies. With Databricks Data Intelligence Platform, ARTC was able to take advantage of cloud-based services like Azure Data Components, and Azure Integration Services, to create a digital ecosystem that could harness the power of data and AI to take Australia’s rail industry into the 21st century.
Making streaming data insights accessible
ARTC embarked upon a business transformation strategy that encompassed the need to digitally modernize and rethink a more efficient way to manage physical rail assets — gaining faster insights from the vast amounts of data it collects, while building resilience to readily respond to customer demands, regulatory compliance and varied world events.
ARTC was dealing with siloed data sets that were required by more than 150 applications generated from different data sources. This meant that any change to the data needed to be altered in several systems, posing inefficiencies and unnecessary complexity, which also impacted the cost of managing a network that is segregated across remote locations in Australia.
"With multiple data sources and legacy systems putting a strain on our teams' ability to operate efficiently, we needed to implement a modern data architecture that would lay the foundation of our digital strategy, and catapult our digital transformation journey," said Andrew Simpson, National Applications Manager, Australia Rail Track Corporation.
ARTC has taken bold, incremental steps as part of its five-year strategy to build a more agile digital ecosystem with data at the helm of its innovation.
Extracting insights and strategies from data to improve decision making
With the help of Databricks, ARTC has successfully launched the first phase of its five-year digital strategy with the implementation of its Enterprise Integration Data Platform (EIDP) built on a cloud-based data lakehouse architecture on Microsoft Azure. The EIDP has enabled the business to unify its approach to data management, advanced analytics and ML-powered solutions.
With the democratization of ARTC's data on a lakehouse architecture, data can now be extracted and decoupled easily, allowing immediate changes to be made securely from a single source of truth — resulting in 30% faster time to insights for freight logistics optimization. Business intelligence tools such as Databricks SQL and Power BI have allowed ARTC's team to extrapolate insights easily in a self-service way, supporting decision making across ARTC's operations.
As an engineering-focused organization, ARTC's data engineers are now able to perform multiple functions in a secure and seamless way — across data ingestion, data preparation and data analysis. By leveraging Databricks Data Intelligence Platform, they have built a data-driven ecosystem where ARTC's systems can not only communicate with each other seamlessly, but implement groundbreaking design patterns with an event-driven architecture and high technical maturity.
Creating API-based big data environments has also prepared ARTC for future events. Seamless access to rail track data provides ARTC with a consolidated view of its track health. This helps to prevent outages and forward-plan track maintenance schedules using predictive analytics, further preventing disruption to operations and network failures.
"Databricks has enabled us to manage the tremendous amount of data from our 8,500km track assets across Australia, and helped us build a strong data foundation that is ready for the future. Not only are our digital aspirations strengthened, but Databricks has also improved the resilience, availability and efficiency of our services, as we continue to support Australia's ongoing economic development," said Anne-Marie, Program Manager, Australia Rail Track Corporation.
Building a strong data foundation for the new era of transportation
ARTC's move to Databricks has also allowed the organization to design frameworks for data modernization, decomposition, and standardization, and enabled it to uplevel its operational agility to address future needs. In addition to achieving higher systems availability, ARTC also saw an 80% improvement in its data ingestion speed, driving toward a proactive decision making approach.
In addition, ARTC has seen progress of its Weighbridge Data Capture (WDC) Sharing project that leverages the new Enterprise Integration Data Platform, ingesting data to inform train locations and weight of cargoes. ARTC sees this as one of many use cases it is able to scale out and deploy at speed, leveraging data to support its use cases of the future.
"Our partnership with Databricks has led to tremendous improvements in the modernization of our networks. Not only are we able to ingest data faster, but we also have a better ability to respond to critical demands from our network in any major event. We are pleased to have been able to collaborate with Databricks to deliver these projects so quickly, and we look forward to empowering more use cases in the future" said Harsh Mishra, Enterprise Architect – Data, Systems and Integration, Australia Rail Track Corporation.
