Nederlandse Spoorwegen (Dutch Railways) operates one of the most heavily used rail networks in Europe, serving more than 1.1 million passengers per day. To maintain safety, efficiency and reliability across a highly complex system, Dutch Railways processes more than 100 billion real-time telemetry data points daily from thousands of onboard sensors. However, legacy systems were primarily designed for historical analysis, making it difficult to detect potential train and infrastructure failures and respond to disruptions before they impacted service. With Databricks, the organization built a unified data platform powered by Real-Time Mode, enabling predictive maintenance, operational intelligence and faster response to disruptions at national scale.
From historical analytics to real-time rail operations
For Nederlandse Spoorwegen (NS / Dutch Railways), operational safety, efficiency and reliability is critical. Even minor equipment failures or weather-related disruptions can quickly ripple across the rail network, causing delays and operational challenges that impact passengers far beyond the original incident. Every train is equipped with 10K+ sensors, generating approximately 36 million data points per hour per train. Across the fleet, this results in more than 100 billion data points per day.
Databricks had already proven its ability to handle this volume for their analytics workloads, but the company needed to go beyond analytics to deliver real-time operational insights. They sought to evolve their platform from an analytical foundation to an operational backbone that could deliver critical data to the business with sub-second latency.
“We were already using data to better understand failures and optimize maintenance, but some failures happen before the data is even processed,” says Gerrit van den Brink, Product Owner of Data, Innovation and Analytics, Nederlandse Spoorwegen. “With real-time capabilities, we can now start acting on issues as they emerge during operation, opening up an entirely new class of use cases that were previously out of reach.”
Databricks Real-Time Mode for streaming intelligence
Dutch Railways modernized its data architecture by adopting Databricks as its unifying foundation, with Real-Time Mode (RTM) for Structured Streaming serving as the core of its operational intelligence layer and enabling real-time decision making.
By moving from micro-batch processing to real-time with RTM, Dutch Railways eliminated architectural complexity while significantly improving performance. The same business logic was preserved, but execution shifted to a continuous streaming model within Databricks, enabling more consistent and scalable real-time operations. As Gerrit van den Brink explains, ‘Normally, we would need a dedicated software engineering team just to get started with real-time analytics. With Databricks RTM, our existing data engineering team can deliver both analytics and real-time data end-to-end on one platform, without changing people, skills, or underlying business logic.”
Alongside RTM, Dutch Railways uses Delta Lake to provide a reliable and scalable data foundation, Databricks SQL to support analytics and transformation workloads and Unity Catalog to ensure governance, lineage and secure access across teams. Together, these capabilities have simplified the architecture and reduced fragmentation between data engineering, analytics and operational teams.
The shift to a unified platform has also changed how teams work. Instead of maintaining separate tools for streaming, analytics, and governance, Dutch Railways now operates within a single environment where data can be accessed, processed, and operationalized consistently across domains. Capgemini supported Dutch Railways in shaping the implementation approach and helping align the platform with broader organizational requirements as it scaled. “This project was truly a team effort. Bringing together our business expertise, Capgemini’s specialized knowledge and the power of the Databricks platform enabled us to deliver the project in record time.,” said Wout de Ruiter, Senior Data and Analytics Consultant at Capgemini.
Ensuring real-time rail reliability
With Databricks in place, Dutch Railways has combined both historical analysis and real-time operational intelligence across its massive rail network within a single platform. The ability to process train telemetry as it is generated gives teams access to insights in real time and unlocks emerging use cases for the business. For example, signals from thousands of onboard sensors per train can now be combined with historical data to potentially identify equipment failure or environmental risks due to ice forming on overhead lines. These capabilities could help operational teams detect issues earlier and coordinate responses more effectively across traffic control and maintenance functions.
This unification of historical and real-time intelligence has driven significant operational cost savings. Consolidating workloads onto Databricks has reduced analytical compute costs by approximately 70 percent. The platform has also accelerated innovation by reducing the time required to onboard new data sources from months to weeks, allowing Dutch Railways to expand its use of operational data beyond traditional telemetry into richer datasets.
The result is a transformation in how Dutch Railways operates its network, enabling rapid response times and greater agility while delivering a more reliable travel experience for the millions of passengers who depend on its services each day. With fewer disruptions and more trains running on schedule, Dutch Railways is helping people reach their destinations more reliably across the Netherlands.
Looking ahead, Dutch Railways is expanding its use of real-time intelligence to even more areas of the rail system—bringing additional operational signals, IoT sensor data, machine logs, geolocation data, weather feeds and third-party network information into Databricks to further strengthen predictive maintenance and network optimization. As the platform continues to scale, the focus is shifting from improving individual operational processes to building a fully adaptive rail system that can sense, predict, and respond to changing conditions across the entire network in real time.
To dive deeper into the architecture they engineered on Databricks, watch their recent Data + AI Summit session: "Streaming at Scale With Real-Time Mode: Sub-Second Train Telemetry Across the Netherlands."