Combatting the impact of financial fraud

Overcoming huge data volumes and diverse operational hurdles

The financial industry in Brazil has faced a persistent and evolving landscape of fraudulent activities, fueled by the rapid digitization of transactions and the increasing sophistication of cyberthreats. In response, AME Digital has been focused on leveraging data and AI to ensure the security and trust of their vast customer base by efficiently detecting and preventing potentially fraudulent transactions before they can cause damage. With AME Digital sales volumes quadrupling during peak season in retail, this critical issue was exponentially more difficult for the company to tackle.

AME faced immense hurdles managing over 700 datasets that were often isolated in data silos within different teams and systems. As AME grew their customer base, the amount of data continued to increase, resulting in workflows that were error-prone and resource-intensive. Being a new, digitally focused team within an older, established corporate structure made it more difficult to align on data objectives and operations and created friction between teams and departments. This lack of cohesion in data integration and collaboration hindered the engineering team from delivering value and fostering innovation, including leveraging data science and machine learning to implement predictive solutions to get ahead of bad actors.

This fragmentation not only impacted AME’s operational efficiency and future potential but also stymied the company's strategic decision-making abilities and agility in responding to market changes and demands. It also created regulatory challenges in the handling of their customers’ sensitive financial data as the fintech company strove to ensure uninterrupted communication with their central bank in compliance with regulatory standards. All of these factors caused a sharp rise in operational expenses, driven primarily by escalating storage and compute costs. What complicated matters most was the existing legacy data infrastructure, based on an on-premises Hadoop environment, which posed roadblocks due to its complexity, high costs and limited scalability — bringing innovation to a standstill.

“Once we grew, we needed to expand the data structure for fraud detection across a wider set of use cases. Ultimately, we chose the Databricks Platform to help us in this complex journey because of its unified environment,” explained Filipe Barbosa, Data Product Manager at AME Digital.

Unifying data to unlock the potential of ML

The Databricks Data Intelligence Platform emerged as a transformative solution for AME Digital. Using Delta Lake, the optimized foundational storage layer within the platform, AME has successfully centralized over 700 datasets. This is crucial for tackling fraud and enhancing analytics to lower costs — two of the company’s main objectives. Filipe Barbosa reasoned, “It's just easier that we have all the tools we need in one place to do every task — without bottlenecks.” The AME team now uses MLflow to manage various machine learning cycles, including an algorithm that helps them catch possible fraudulent behavior while reducing their number of manual reviews. With the help of Eleflow Big Data, a Databricks certified partner, AME has leveraged the Databricks Platform to analyze large volumes of data in real time, identify suspicious patterns of behavior and block possible fraudulent transactions before they occur.

Building on Delta Lake and MLflow, the integration of Unity Catalog further bolstered AME Digital's ability to manage and secure its data. Unity Catalog centralized the management of personally identifiable information (PII) and enhanced access controls to ensure better compliance and governance. Due to the ease of use and smooth implementation, AME further expanded their Databricks collaboration with Power BI integration. By effectively harnessing the robust data processing capabilities of the Databricks Platform, AME was able to feed a wide range of data into Power BI and create more than 700 dynamic dashboards for finance and product development decision-making. This integration not only streamlined reporting processes but also significantly elevated data visibility and usability throughout the organization. With Photon, the highly optimized Databricks query engine designed to enhance performance for large-scale data processing, AME Digital began to experience massive time savings across departments.

None of this could’ve happened without Eleflow's expertise in migrating and efficiently managing growth from 1TB to over 400TB of diverse data. The strategic support of this implementation partner, combined with Databricks advanced features, optimized AME data workloads for performance and cost efficiency. All in all, the solid partnership between AME Digital, Eleflow and Databricks has laid the groundwork for AME's future ambition — establish a center of excellence (CoE) designed to democratize data to encourage self-service, foster data-driven decision-making and reduce the need for extensive engineering resource investment in building data pipelines and dashboards.

Evolving operations, fraud detection and cost management

In AME Digital’s pursuit of operational efficiency and cost management, Databricks gave the fintech company everything they needed to achieve their initial objectives from a single platform. First, the data team’s relentless efforts to implement Databricks across different use cases, including data management, data governance and machine learning development, led to a 34% reduction in operational costs.

Second, Databricks facilitated diverse, large-scale data workloads through the use of Delta Lake, Unity Catalog, Photon and the Power BI integration, resulting in considerable time savings across various business functions. Notably, job execution times were cut from 5 hours and 30 minutes to a mere 50 minutes — an 85% time reduction in data workflows.

Third, the impact of Databricks was particularly evident in fraud prevention, with AME Digital’s fraud detection model achieving a 90% accuracy rate. For Helena Valente, Data Strategist at Eleflow, “Working with Databricks has been a joyride. Seeing the growth of AME’s data foundation from 1TB to 400TB+ — and what they could create to make that data impactful to their organization — is truly an incredible feat.” AME feels more confident than ever, especially during high-demand periods such as Black Friday, that their teams can get ahead of fraudulent behavior before it impacts customers and trust the platform to execute all jobs and pipelines, regardless of the number of orders made.

Looking to the future, this strategic integration of data, analytics and AI within AME Digital business operations positions the brand for continued growth and innovation. Barbosa concluded, “As long as Databricks continues to innovate, we are confident that we can promote our business as a digital leader by simply using their toolset.”