Session
Sponsored by: Capital One Software | How Capital One Balances Lower Cost and Peak Performance in Databricks
Overview
Experience | In Person |
---|---|
Type | Lightning Talk |
Track | Data Warehousing |
Industry | Enterprise Technology, Health and Life Sciences, Financial Services |
Technologies | Databricks SQL |
Skill Level | Intermediate |
Companies need a lot of data to build and deploy AI models—and they want it quickly. To meet this demand, platform teams are quickly scaling their Databricks usage, resulting in excess cost driven by inefficiencies and performance anomalies. Capital One has over 4,000 users leveraging Databricks to power advanced analytics and machine learning capabilities at scale. In this talk, we’ll share lessons learned from optimizing our own Databricks usage while balancing lower cost with peak performance. Attendees will learn how to identify top sources of waste, best practices for cluster management, tips for user governance and methods to keep costs in check.