by Sara Steffen
For over 40 years, Thomas’ central ethos has been that companies can elevate job satisfaction and productivity by better understanding how people interact with each other in the workplace. As they sought to expand and deepen their impact in the field of people science, Thomas encountered major roadblocks with traditional data management and utilization approaches. They turned to Databricks for help with improving their data workflows — from ingestion and transformation to analysis and solutions for customers and employees. Plus, the integration of GenAI through AI Search and retrieval augmented generation (RAG) further improved content searchability and automated content generation.
Thomas’ foundational psychometric assessment model — rooted in paper-based processes and designed for one-on-one interpretations — became increasingly limiting as they grew their customer base. Not only did these traditional methods struggle to scale, but they weren’t connected to the typical applications necessary within a modern work environment.
Recognizing the urgent need to mature and grow as a company, Thomas embarked on a path of digital transformation. One main objective was to transition away from the labor-intensive model of manually training HR directors and hiring managers to understand and implement their psychometric assessments. Thomas wanted to find ways to give more employees access to people science and make their tools easier to use by customers.
They also recognized the possibility of saving their own employees valuable time when providing personalized feedback and suggestions to customers. According to Director of Data Science Dr. Luke Treglown, this was easier said than done: “Our legacy platform had millions — to the point of billions — of words of content because it included every possible iteration of how the content could be written and personalized to the specific recipient. While you think this would be helpful, translating evergreen content into useful responses for each client became an absolute nightmare.”
Addressing these challenges became a key priority. Because a user profile is completed with Thomas every 90 seconds, it was imperative for them to be able to ingest massive amounts of data more effectively and maintain their leadership in the field of people science.
Treglown stated, “We were still struggling with how to guide people to the pieces of content that were needed to solve their problem. By adopting Databricks, we transformed our data handling capabilities, enabling us to manage and analyze larger volumes of data more efficiently and extend our reach into new markets.”
The Databricks Data + AI Platform and Databricks tools enabled Thomas to improve their data workflows — from ingestion to transformation to analysis — all within a secure environment. Plus, the platform’s integration with generative AI tools like retrieval augmented generation (RAG) and natural language processing (NLP) delivered even more benefits in content management, searchability and personalization.
Using RAG techniques to prompt LLMs via Databricks AI Search gives Thomas the ability to rapidly get insights from their extensive database. Additionally, this GenAI integration enabled their users to propose queries and receive automatically generated, detailed and tailored insights from unstructured data. Treglown explained, “Databricks AI Search sparked a breakthrough for us. We could give our clients the ability to find the answers they need, rather than throwing a 40- to 50-page report at them. This made our content incredibly more dynamic and interactive.”
Databricks provides a secure environment that allows Thomas to leverage their large datasets and AI capabilities without compromising on their ethical commitments to customers. The platform’s built-in features for managing data access and its ability to seamlessly integrate with existing security protocols mean that sensitive information is always protected, and data integrity is never compromised. Treglown shared, “With Databricks, GenAI is not a black box. We can talk to our customers and take them step-by-step through how we got the insights while ensuring their data is handled with the utmost care.”
By automating and personalizing the evaluation process, Databricks tools from Databricks have transformed how Thomas approaches psychometric assessments. Thanks to GenAI, each client receives insights that are most relevant to their specific contexts and needs, enhancing the overall impact of Thomas’ services and forming the foundation of their new “Perform” product.
Databricks also gives Thomas the ability to integrate their new, AI-driven processes into popular platforms like Microsoft Teams. Now, their services are available within workflows clients already use. This seamless integration helps maintain continuous engagement and ensures that Thomas’ tools are adding value without disrupting the daily operations of businesses.
With AI-driven enhancements, the user experience of the platform has become significantly more interactive, personalized and efficient for locating content and other information. This new iteration has increased user satisfaction and led to deeper engagement.
During Thomas’ transition from traditional, paper-based methods to a cutting-edge data solution, their organizational culture has quickly embraced innovation and continues to push the boundaries of what’s possible in people science. Concluded Treglown, “With Databricks, we were able to go from proof of concept to minimum viable product in weeks. And now that we have a solid, unified foundation, we can go where we want to with AI.” In fact, Thomas has been able to integrate GenAI into three different platforms within the last three months.
Treglown summed up the impact to Thomas: “We needed to transform as a business because doing more of the same wasn’t going to help us grow. How do we shake up and truly innovate what we’re doing? For us, GenAI was the obvious way to do that.”
See more customer success stories here.
Thomas' original psychometric assessment model relied on paper-based processes and one-on-one interpretation, which made it hard to scale as the customer base grew and left the tools disconnected from applications used in modern workplaces. The legacy platform also held millions to billions of words of content covering every possible personalized iteration, making it extremely difficult to translate that evergreen content into useful responses for each client. Because a user profile is completed with Thomas every 90 seconds, the company needed a way to ingest and manage much larger volumes of data more effectively.
Thomas uses retrieval augmented generation (RAG) techniques to prompt LLMs via Databricks AI Search, allowing users to submit queries and receive automatically generated, detailed, tailored insights from unstructured data. According to Director of Data Science Dr. Luke Treglown, this let clients find the answers they need instead of receiving a static 40- to 50-page report, making Thomas' content far more dynamic and interactive. The same Databricks tools also support ingestion, transformation and analysis of Thomas' data within a secure environment.
Perform is the new product Thomas built on top of its GenAI-driven, automated and personalized psychometric assessment process. By automating and personalizing the evaluation process, Databricks tools help each client receive insights most relevant to their specific context and needs, which forms the foundation of Perform. Thomas has also integrated this AI-driven process into platforms clients already use, such as Microsoft Teams, so the service fits into existing workflows.
Databricks gives Thomas a secure environment with built-in features for managing data access, so the company can use its large datasets and AI capabilities without compromising its ethical commitments to customers. These features integrate seamlessly with Thomas' existing security protocols, keeping sensitive information protected and data integrity intact. Treglown noted that with Databricks, GenAI is not a black box, since Thomas can walk customers step-by-step through how insights were generated while handling their data with care.
Thomas moved from proof of concept to minimum viable product in a matter of weeks after adopting Databricks, and has since integrated GenAI into three different platforms within the last three months. The AI-driven enhancements have made the platform more interactive, personalized and efficient for locating content, which has increased user satisfaction and led to deeper engagement. Treglown framed the shift as a necessary business transformation, saying GenAI was the obvious way for Thomas to innovate rather than simply doing more of the same.
Subscribe to our blog and get the latest posts delivered to your inbox.