Ensuring corporate audits are smooth sailing

Manual audit processes impede efficiency and accuracy

Hapag-Lloyd is a leading global liner shipping company with a rich history and a commitment to providing efficient, reliable and sustainable transportation services. With a vast network spanning over 400 offices in 140 countries, Hapag-Lloyd operates a fleet of 280 modern ships that transport 11.9 million TEUs (twenty-foot equivalent units) per year. The company leverages cutting-edge technologies to maintain a customer-first approach. For example, by equipping their fleet of 1.6 million containers with real-time tracking devices, Hapag-Lloyd can provide clients with valuable data for recommendations and business decisions.

In pursuit of continued operational excellence, Hapag-Lloyd identified the need to optimize their corporate audit processes. “Over a couple of years, it was already our aim to reduce time spent on documentation and report writing,” explained Ulrich Daniel, Director of Corporate Audit Analytics. “As this process involved several manual instances, the key was to get the quality on a very high level.”

The primary use cases for leveraging generative AI included enhancing the efficiency and accuracy of audit finding generation and executive summaries. By automating the generation of findings and creating a more streamlined documentation process, Hapag-Lloyd sought to reduce the time auditors spend on manual tasks, allowing them to focus more on critical analysis and decision-making.​

The integration of generative AI aimed to revolutionize how auditors interact with vast amounts of data. This included developing a chatbot for process documentation, which enabled auditors to query specific information from numerous files quickly. The goal was to provide a simple, natural language interface that could retrieve precise details from various documents, significantly enhancing productivity and accuracy in various auditing fields.

Before adopting Databricks, Hapag-Lloyd faced several challenges in their auditing and documentation processes. As Daniel mentioned, the traditional methods of generating audit reports were time-consuming and involved numerous manual steps, leading to inefficiencies and potential inconsistencies in documentation. “Our existing infrastructure, including tools like vector databases, did not support the rapid setup and deployment of AI models required for our audit optimization efforts. Ensuring the accessibility of instances was also a challenge. We had an AWS SysOps account, but it was very difficult to set up instances in a fast manner,” said Daniel.

The need for a high-quality, reliable and scalable solution was evident. So they started deploying their generative AI initiatives within the Databricks Data Intelligence Platform. According to Daniel, “All those challenges have been solved by the Databricks team. Compared to our AWS SysOps account, we could get our instance setup far leaner with Databricks. In terms of costs, it improved step-by-step.”

Rapidly building GenAI prototypes with Databricks Mosaic AI

Databricks facilitated the deployment of advanced AI models tailored to Hapag-Lloyd’s specific requirements. Mosaic AI made it easy for them to evaluate various models and choose the right one for their use case based on price/performance characteristics. “Initially, models such as Llama 2 70b and Mixtral were tested, but ultimately, we ended up using the Databricks DBRX model. DBRX returns far better results than the previously tried models,” explained Daniel. This model, a transformer-based decoder-only LLM, was pretrained on extensive datasets, making it suitable for generating high-quality audit findings and summaries​. Daniel’s team dubbed this prototype the Finding Generation Interface.

Hapag-Lloyd engineers then fine-tuned Databricks’ open source DBRX model on 12T tokens of carefully curated data. The solution architecture included a series of steps, starting with data ingestion, preparation and prompt engineering and ending with model evaluation and deployment. The architecture enabled seamless integration with existing data pipelines and provided a robust framework for continuous improvement. The process involved generating findings, summarizing results and storing them in a Delta table for easy access and retrieval​.

Using Databricks MLflow, Hapag-Lloyd was able to automate the evaluation of prompts and models. This reduced the time-consuming manual process of generating and reviewing audit findings. The platform’s capabilities in managing the full ML lifecycle — from data ingestion to model deployment — played a crucial role in streamlining the audit process. “Databricks delivers amazing support through the full lifecycle,” added Daniel.

To further enhance efficiency, a chatbot interface was developed using Gradio and integrated with Mosaic AI Model Serving. This chatbot allowed auditors to query specific information from documents using natural language, significantly reducing the time spent searching for data. The chatbot used retrieval augmented generation (RAG) to provide accurate and contextually relevant responses, further improving the audit process​.

Databricks’ comprehensive suite of GenAI tools and services facilitated better collaboration among teams and made it easier to access and manage audit data. This was particularly important in a global organization like Hapag-Lloyd, where efficient data management and collaboration are critical​. “It definitely speeded up the process,” confirmed Daniel. “Without Databricks, it would have taken far longer to get the required expertise.”

Enhancing audit efficiency with GenAI

Databricks’ GenAI capabilities empowered Hapag-Lloyd to enhance their audit efficiency significantly, providing a scalable and high-quality framework for audit and documentation processes. Auditors now spend only 5 minutes creating a new finding from bullet points, down from 15 minutes. This represents a 66% decrease in review time per finding — a huge time savings given that there are, on average, seven findings per audit. The time required to review each executive summary has decreased by 77%, from 30 minutes to just 7 minutes.

Building the Finding Generation Interface and chatbot through Databricks not only streamlined operations but also set the stage for future advancements in audit automation and efficiency​. Hapag-Lloyd plans to extend the current solution to cover more aspects of the audit process. This includes fine-tuning large language models to better structure and organize audit reports, further reducing the time and effort required from auditors. The company is looking to improve and automate the evaluation process using the Mosaic AI Agent Evaluation framework. This will streamline the assessment of audit findings and ensure consistent quality across all reports.

“Moving forward, generative AI will have a significant role in freeing colleagues from administrative tasks. Databricks is the partner of choice for that,” concluded Daniel.