Building the foundation for the 5G revolution

Building the infrastructure for scalable 5G deployment

Digital Nasional Berhad (DNB) was established by the Malaysian government to accelerate 5G infrastructure deployment across the country. With an ambitious goal of delivering 80% 5G coverage in populated areas by 2024, DNB achieved this milestone in 2023, a year ahead of schedule, making it one of the fastest 5G rollouts globally.

The rapid deployment required a strategic, data-driven approach to project management, network planning, optimization and real-time cost monitoring. Data helps track progress and costs, enabling efficient resource allocation and budget adherence. Real-time data plays a crucial role in network optimization, improving 5G performance and, ultimately, user experience. The key challenge was managing vast amounts of real-time data, including determining next actions and analyzing performance metrics, coverage gaps and service quality indicators.

DNB processes vast amounts of data daily from their nationwide 5G network, necessitating a scalable, high-performance data platform. Initially, DNB evaluated multiple technologies before selecting the Databricks Data Intelligence Platform. “Databricks provided the flexibility we needed to optimize, compute and process data efficiently without impacting storage costs,” Khaidyr Abdul Rahman, Head of the Data Center of Excellence (CoE) at DNB, said.

Empowering collaboration and innovation with a unified data platform

The Databricks Data Intelligence Platform provided a range of advanced tools and capabilities that have transformed how the organization managed and utilized their data. One of the most impactful solutions was the adoption of Delta Lake. Through a medallion architecture, DNB could organize their data into layers: raw data ingested into the Bronze layer, cleaned and structured data in the Silver layer and refined data in the Gold layer — making the data ready for analytics and machine learning. This structured approach allowed for streamlined workflows and better data quality management. “The medallion architecture has been instrumental in helping us organize our data workflows. It ensures that each layer — from raw data to refined datasets — is optimized for specific use cases,” Mark Edward, Data Platform Engineering Manager at DNB, said.

Databricks Workflows became another key pillar of DNB’s data strategy, replacing traditional orchestration tools. Workflows enabled DNB to automate data pipelines across their network operations and other business units. The real-time monitoring and alerting capabilities built into Workflows have provided greater visibility into data processes, helping the team quickly address any errors or bottlenecks.

Serverless computing has further optimized the platform by dynamically allocating resources based on workload requirements. This has eliminated the need for overprovisioning, particularly for smaller or ad hoc tasks, resulting in significant cost savings. Serverless has also enhanced the flexibility of DNB’s operations, enabling teams to quickly respond to new use cases or changing business needs without the overhead of traditional infrastructure management.

Databricks Notebooks has improved collaboration between teams, by allowing data scientists, engineers and analysts to work together seamlessly within the same environment. This collaborative functionality has not only increased productivity but has also ensured consistency across projects. The integration with Power BI has further empowered business users by providing them with direct access to data-driven visualizations and dashboards, enabling intelligent decisions without requiring technical expertise. “Databricks has made it easier for our teams to work together and innovate,” Khaidyr explained. “With all our data processes centralized and optimized, we can now focus on higher-value tasks, such as developing AI-driven solutions for our business units.”

Databricks’ Unity Catalog played a crucial role in governance and security. By centralizing data access controls and providing clear lineage, Unity Catalog ensures that sensitive information is handled appropriately. This transparency and security reassure stakeholders across the organization, making it easier to align on data strategies and policies.

Collectively, these tools have allowed DNB to build a data platform that is not only robust and scalable but also cost-effective and user-friendly. This foundation has enabled the organization to begin building advanced analytics and AI solutions to support their 5G mission. For example, DNB has started using MLflow to manage the complete lifecycle of their machine learning models. From tracking experiments and managing feature stores to deploying models into production, MLflow ensures that DNB’s machine learning workflows are efficient and scalable.

Fueling future innovation with a cost-optimized data strategy

DNB’s use of Databricks has led to significant business and operational improvements. The organization has reduced operational costs by up to 70%, ensuring that high-volume data processing could be performed without unnecessary expenses. This cost efficiency was critical for DNB, given the scale and pace of their 5G rollout, enabling the team to allocate resources more effectively across their initiatives.

Performance improvements were another notable outcome of leveraging Databricks. By transitioning from traditional orchestration tools to Databricks Workflows, DNB achieved a 60%–70% increase in data pipeline performance speeds. This acceleration allowed for faster data insights, which directly supported decision-making across various departments, from network operations to finance.

AI and advanced analytics have become a central focus for DNB, empowered by the solid data platform established with Databricks. The organization has begun integrating geospatial analytics and AI-driven assistants into their operations. For example, DNB is building AI-powered chatbots that provide network engineers with real-time access to performance insights, paired with geospatial visualizations of base station coverage and heatmaps. This innovation ensures that teams can quickly identify and address network issues while optimizing future deployment strategies. “Databricks has empowered us to scale and innovate while maintaining cost and operational efficiencies,” Khaidyr said. “It has given us the foundation we need to build next-generation AI and analytics solutions that drive our 5G mission forward.”