Product descriptions:
Experian is a multinational data broker and consumer credit reporting company with the end goal of helping their customers during big life moments — from buying a home to helping their kids with college. Experian also sees themselves as a great equalizer and focuses on financial inclusion. Using technology to create better opportunities for people and organizations globally, Experian wanted to improve their internal and customer-facing GenAI initiatives. The team had ambitions to build an AI chatbot, specifically to respond to contact center emails. Experian faced increasing pressure to enhance customer support efficiency while maintaining high-quality service. The existing systems were not scalable to meet the growing volume of customer inquiries, leading to longer response times and potential customer dissatisfaction. With Databricks, Experian built “Latte,” a GenAI-powered chatbot that increased their customer net promoter score (NPS) — a metric used to measure customer loyalty and satisfaction — by 8%.
Transforming a chatbot into an email engine
With over 22,000 employees operating in 32 countries, Experian is a truly global company that believes in financial power for all. Trusted with the data of 1.1 billion people and 150 million active businesses worldwide, the multinational data broker and financial credit reporting company wanted to leverage this treasure trove of consumer information to make the customer experience even more intuitive and responsive. The team set out to host an open source GenAI model in a protected environment to safeguard customer data and improve productivity and customer satisfaction. They eventually succeeded by fine-tuning a Llama 8B model to power a GenAI chatbot. By automating 1,000+ daily customer emails, Experian’s goal was to eventually phase out the manual User Acceptance Testing (UAT) review process and allow more than 35% of emails to be handled autonomously by their GenAI chatbot, Latte.
“Once we could prove our GenAI models were stronger for text-based email automation, we could repurpose the initial call-based AI respondent for the customer support center through a more scalable, text-forward approach,” James Lin, Head of AI/ML Innovation at Experian, explained. This new GenAI chatbot would aim to educate consumers on factors influencing their credit scores, like payment history and credit utilization, to improve financial literacy. However, building such powerful, future-looking GenAI solutions required a strong data foundation, which meant that Experian had to address several technical challenges as they scaled their AI capabilities.
First, Experian needed to manage large volumes of structured and unstructured data at scale. Unfortunately, much of their existing documentation and data sources weren’t fully GenAI-ready, with issues in quality and structure. Second, their previous cloud infrastructure created limitations — services were complex, slow and lacked key features, like native LLMOps and Vector Search capabilities. On top of that, fine-tuning models took roughly 86 hours.
Because of these obstacles, building a proper LLMOps pipeline became essential for Experian. Yet, the brand needed integrated capabilities — instead of stitched-together services — for fine-tuning, model serving, governance and scalability. If this weren’t enough, they also faced strict regulatory standards, requiring complete traceability of model inputs, outputs and synthetic data generation. This made flexibility especially challenging, and the Experian team found themselves in need of a model-agnostic environment that would allow them to easily experiment and switch between different open source models to support rapid innovation. Luckily, they found an all-in-one solution in Databricks to remedy their broad range of challenges.
Delivering faster, more efficient support at scale
The Databricks Data Intelligence Platform helped Experian consolidate GenAI development while supporting critical goals, like flexibility, speed, cost control and governance. They started with Databricks’ Mosaic AI to build, deploy, evaluate, monitor and govern AI agents, using it to fine-tune a Llama 8B model, Latte, to automate email responses, with the goal of reducing manual workloads, accelerating reply times and improving overall customer engagement. Unlike traditional MLOps pipelines, this effort required a GenAI-specific architecture, LLMOps, to support the complexity of fine-tuning, along with versioning and compliance. Databricks offered these capabilities natively, which allowed Experian to manage their model lifecycle without cobbling together multiple third-party tools.
To support model fine-tuning, the popular credit reporting brand initially generated synthetic data using MPT (Mosaic Pretrained Transformer) models but transitioned to DBRX, Databricks’ open source large language model (LLM), to create richer instruction datasets. This shift improved prompt handling and model accuracy, giving their teams greater flexibility to experiment with different prompts, refine outputs and adapt quickly as GenAI initiatives evolved. James noted, “During early testing, Databricks delivered faster LLM response times and six times the initial token throughput at the same hourly cost, compared to their cloud services. Even after optimizations, Databricks maintained a 3x performance advantage, all without compromising model accuracy or compression.”
Another tool Experian activated within their comprehensive Databricks toolbox was Vector Search. Instead of having their GenAI models sift through documents manually or rely on keyword matches, Databricks’ Vector Search capabilities helped the models quickly retrieve the most relevant information based on the meaning of customer chat queries. Vector Search powered their retrieval augmented generation (RAG) pipelines. Compared with their initial ChromaDB setup, it delivered faster response times and better retrieval quality. For example, if a customer asked “How do I freeze my credit?” or “How can I lock my report?”, the model could recognize the intent behind both questions and retrieve the right content, even when phrasing differed. Building use-case-specific vector stores and preparing both structured and unstructured source documentation through Databricks data pipelines strengthened the underlying database, filling content gaps and ensuring data was fully GenAI-ready.
For deployment, Experian combined Mosaic AI Model Serving with their own AI Gateway. This gave their teams flexible control over how models were hosted and accessed internally. Governance was managed through Unity Catalog and MLflow, enabling full traceability across the model lifecycle — including synthetic data lineage and fine-tuning inputs — while maintaining both compliance and innovation speed. Before full production rollout, Agent Evaluation was used to continuously test model outputs against internal benchmarks and ensure consistent quality. To further validate model reliability, the team used AI Functions to classify incoming support questions by topic and identify which categories could be confidently automated to build stakeholder trust in Latte’s performance.
Turning early GenAI wins into lasting business value
The success of Latte marked a major milestone in Experian’s contact center operations. By fine-tuning a Llama 8B model with Mosaic AI, the team transformed an AI chatbot into a production-ready email automation tool. Initially launched with a human-in-the-loop to validate responses, Experian is moving closer to full automation, with Latte now handling over 35% of incoming customer emails — freeing up teams to focus on more strategic, high-value initiatives that drive business growth. With the ability to interpret multipart questions, understand customer intent and respond with clear, accurate information, Latte has helped Experian improve the customer experience.
In fact, customer satisfaction reflected these positive changes, with an 8% lift in NPS following the rollout of GenAI-generated email responses. Additionally, model fine-tuning time dropped from 86 hours to just 8 hours on Mosaic AI — a 91% reduction — and improved even further in production, with some runs completing in under an hour at a cost of approximately $100.
Behind the scenes, Latte has given the customer experience team more confidence and flexibility. James concluded, “Since employees can now focus on more complex customer needs, internal job satisfaction has improved and, more importantly, without any loss of human jobs. Building on this success, Experian plans to increase automation of the email channel to 50% and expand the use of Latte to other customer service channels.”
Finally, Latte laid the foundation for what’s next for Experian. With dozens of GenAI use cases documented for future delivery, this early achievement has become a model for how AI and automation can contribute to companywide objectives and key results (OKRs), from improved employee sentiment to heightened customer trust. Together, these Databricks components have provided Experian with a secure, scalable framework to confidently deploy and monitor Latte and future GenAI solutions across their organization.