“The usage of Databricks over the years has broadened significantly. We started out using Databricks as a big data and AI platform but the scope has broadened. We have an entirely different class of citizen engineers and data scientists who are using it as a modern business intelligence tool to make smarter business decisions.”
Digitalisation will play a vital role in helping the energy industry to tackle climate change. It is helping Shell to accelerate through the energy transition. Across the business, they are turning to data and AI to improve operational efficiencies, drive customer engagement, and tap into new innovations like renewable energy. Hampered by large volumes of data, Shell chose Databricks to be one of the foundational components of its Shell.ai platform. Today, Databricks is part of Shell’s solution to enable hundreds of Shell’s engineers, scientists and analysts to innovate together as part of their ambition to deliver cleaner energy solutions more rapidly and efficiently.
The challenges of extracting insights at scale
Throughout its 100-plus-year history, Shell has generated pioneering ideas that have influenced the way we consume energy.
“We, as an industry, are going through a massive transition,” explained Dan Jeavons, VP Computational Science and Digital Innovation at Shell. “Digital technology is absolutely core to making our existing business more effective and efficient. As the industry continues to expand into new areas of energy that are more sustainable and reduce environmental impact, data and digital technology are now table stakes.”
While digital transformation is a primary initiative for Shell, challenges remain with legacy technology infrastructure, the complexities of an exponential growth in data, and the lack of data engineering and science skills needed to build data-powered solutions.
Shell has met these challenges head-on by creating a Digital Innovation and Computational Science capability, where teams continually work to identify the highest value use cases across the entire value chain. However, although they were identifying opportunities to innovate with data, Shell had the challenge to scale its data infrastructure for analytics, big data processing and machine learning.
Unifying data and AI across the enterprise to fuel innovation
Shell chose the Databricks Lakehouse Platform as one of the key tools within the Shell.ai Platform. Databricks provides Shell’s data team with a scalable, fully managed platform that unifies all their data, analytics and AI workloads. The interactive workspace has not only democratized access to data but has supported cross-team collaboration across data engineering, data science and the analyst teams.
“Shell has been undergoing a digital transformation as part of our ambition to deliver more and cleaner energy solutions. As part of this, we have been investing heavily in our data lake architecture. Our ambition has been to enable our data teams to rapidly query our massive datasets in the simplest possible way. The ability to execute rapid queries on petabyte scale datasets using standard BI tools is a game changer for us. Our co-innovation approach with Databricks has allowed us to influence the product roadmap and we are excited to see this come to market,” said Dan.
This low barrier of entry has opened up analytics beyond machine learning, including business intelligence and reporting. In fact, Shell’s focus on data and analytics has enabled over 250 data analysts (or citizen data scientists), and 800 citizen data scientists to be more productive with all the data available to them.
Transforming Shell into the energy company of the future
Shell’s digital innovation and IT teams are now able to explore and deploy new data-driven solutions focused on improving supply chain operations as well as unlocking high-valued use cases that bring to life differentiated capabilities for their customers and their own businesses.
From an operations perspective, one of the biggest challenges any major industrial company faces is efficiently managing its inventory and supply chain. Shell stocks thousands of spare parts across its global facilities, and its inventory analysts were struggling to understand what level of spare parts they should hold in their warehouses. With Databricks as part of its digital solutions, Shell was able to leverage its full historic data set to run 10,000+ inventory simulations across all its parts and facilities. Shell’s inventory prediction models now run in hours rather than days, significantly improving stocking practices and saving a lot of money annually.
Shell has also developed a recommendation engine for its new loyalty program used by millions of customers. Running on Databricks, the AI software can look at the full transaction history of a customer and use the information to tailor the offers and rewards to the preferences of the individual, combining their data with other aggregated data.
A data-driven culture that delivers results
Today Shell is redefining its boundaries of the oil and gas industry through data and AI. With Databricks as a key component of the Shell.ai platform, Shell is able to run data analytics and deploy machine learning models that improve operational efficiencies.
Using a common platform has empowered engineers, data scientists and analysts to be more agile, collaborative and data driven. Shell currently has over 100 AI projects running, and it’s only just getting started. In the coming years, Shell aims to make leaps in technological advancements powered by data and AI — from trillions of IoT sensors all generating data to 3-D printed equipment and parts that aims to change the global supply chain and reduce costs—and Databricks is part of the Shell.ai platform that will help make this a reality.