Session

Sponsored by: Capital One Software | How Capital One Balances Lower Cost and Peak Performance in Databricks

Overview

ExperienceIn Person
TypeLightning Talk
TrackData Warehousing
IndustryEnterprise Technology, Health and Life Sciences, Financial Services
TechnologiesDatabricks SQL
Skill LevelIntermediate

Companies need a lot of data to build and deploy AI models—and they want it quickly. To meet this demand, platform teams are quickly scaling their Databricks usage, resulting in excess cost driven by inefficiencies and performance anomalies. Capital One has over 4,000 users leveraging Databricks to power advanced analytics and machine learning capabilities at scale. In this talk, we’ll share lessons learned from optimizing our own Databricks usage while balancing lower cost with peak performance. Attendees will learn how to identify top sources of waste, best practices for cluster management, tips for user governance and methods to keep costs in check.