Driving eco-innovation in the energy sector for a cleaner future

Struggling with eco-focused mission due to lack of automation

OMV’s commitment to sustainable product development is at the heart of their transition toward becoming an integrated sustainable chemicals, fuels and energy company. This commitment aligns with their focus on circular economy solutions and the need to navigate the intricate landscape of new sustainable processes and European sustainability regulations. However, OMV’s efforts were hindered by challenges in efficiently processing and analyzing vast amounts of regulatory, product certification and internal data. The existing data infrastructure was inadequate for the advanced data processing capabilities required, making it difficult to generate the in-depth analyses crucial for guiding their sustainability initiatives.

Harisyam Manda, Senior Data Scientist at OMV, explained, “Our business wanted faster access to knowledge and a smoother way to synthesize huge amounts of text and tabular data related to rules and regulations from the EU — that was our initial problem statement.”

Compounding these data discoverability challenges, OMV struggled to build efficient data pipelines for batch processing. Even though the company was eager to build this knowledge platform to support their International Sales and Marketing team in gaining access to the latest EU regulations and product certifications, the team was hampered by a poor efficiency-to-cost ratio. These constraints, paired with the high costs of overusing vector search units, made it difficult for OMV to meet their sustainability objectives in a cost-effective and timely fashion.

Additionally, OMV wanted to ensure that their data workflows could scale in line with their growing regulatory needs instead of creating operational bottlenecks that prevented the company from adapting quickly to new demands. Recognizing GenAI’s potential to address these issues, OMV initially explored GenAI’s potential to enhance productivity. However, it became increasingly clear that a comprehensive, flexible data management solution was also necessary to support OMV’s sustainability ambitions.

Tapping into GenAI solutions for processing and discovery

OMV chose to leverage the Databricks Data Intelligence Platform as the core infrastructure for their downstream business operations, enabling them to power data and AI workloads across their organization. The scalability of the Databricks Platform complemented OMV’s ability to process vast amounts of data and integrate Databricks Mosaic AI tools like Vector Search. OMV’s batch ingestion pipeline now utilized BAAI’s BGE M3 model for multilingual embeddings, ensuring that the data processing could handle the complexities of multiple languages across the EU region while indexing 100,000 pages of regulatory and internal documents. Harisyam confirmed, “Databricks significantly facilitated the development of our generative AI solutions by providing Vector Search endpoints, Model Serving endpoints and data ingestion pipelines.”

OMV developed a retrieval augmented generation (RAG) chatbot using the Mosaic AI Agent Framework. Depending on the number of users and system demand, OMV primarily utilized either GPT-3.5 or GPT-4. The internal chatbot was continuously improved through a human feedback loop, which allowed OMV to refine their vector store with FAQ-based queries. Databricks Model Serving endpoints facilitated the deployment and management of scalable, secure and production-ready machine learning models. The RAG agent was also equipped with tools to minimize manual web searches and streamline the search for new regulatory information. Integrating various open source AI models into their operational workflows was crucial for OMV. With Databricks Mosaic AI tools, OMV’s teams could quickly deploy and test these models without extensive script maintenance, making it easier to achieve their new objectives.

To further optimize data workflows and maintain security, OMV used Unity Catalog to manage access control and data security. This tool was particularly important for the new internal tools, like the agentic RAG chatbot, which required stringent data access controls to maintain their integrity. Unity Catalog ensured that only authorized users could access specific documents, and it integrated seamlessly with other Databricks tools to provide a secure and governed environment for data management and GenAI experimentation. This allowed OMV to leverage AI technologies with greater precision and confidence, driving innovation while maintaining strict compliance with regulatory standards.

Boosting efficiency and productivity while controlling costs

By leveraging Databricks, OMV has significantly enhanced efficiency and scalability across their data and AI operations. Now, OMV can easily manage 100,000 pages of regulatory documents, in multiple languages. Databricks Mosaic AI tools have helped OMV cut document and web search times by 20% and reduce the time to incrementally add more documents. Additionally, OMV experienced a 20–25x reduction in cost for their vector search tools.

In addition to improving OMV’s operational efficiency, Databricks has helped OMV transform their approach to data-driven decision-making. The integration of these advanced AI solutions has provided faster, more accurate access to critical information, leading to better-informed decisions across the company. Thanks to Databricks’ robust infrastructure that includes everything from Vector Search and Model Serving endpoints to data warehousing, OMV can now manage and process large volumes of data with increased speed and reliability, paving the way for ongoing AI experimentation.

As OMV continues their AI transformation journey, with sustainable energy as a key focus, the Databricks Platform will remain vital in optimizing data management and AI capabilities. The seamless handling of data and AI workflows has already empowered OMV to synthesize data efficiently and access knowledge, positioning the company to drive innovation and maintain a competitive edge in the energy sector. Harisyam concluded, “Working in this field with Databricks puts you at the forefront of AI research. The ability to implement these technologies at scale is beneficial for our team and the entire business unit.”