Brickster Voices: Meet Chao Cai, Sr. Director, Engineering

Brickster Voices is a series that spotlights the people who make our work possible. Through personal career journeys, behind-the-scenes looks at impactful projects, and a glimpse into how we work together, these stories offer a window into life at Databricks. Whether you’re exploring new opportunities, curious about our work in Data + AI, or simply inspired by stories of growth and collaboration, Brickster Voices invites you to get to know the individuals driving our mission forward.

As part of our Brickster Voices series, Employer Brand Project Manager Andrea Fernandez sat down with Chao Cai, Sr. Director of Engineering, for a candid conversation on innovation, leadership, and the future of AI/BI. Chao leads engineering for the AI/BI product line, overseeing the development of innovative natural language BI experiences such as Genie. His career journey in this space is both innovative and inspirational.

A: Could you share a little bit about your career journey and what led you to engineering leadership and the AI/BI space?

Before joining Databricks, I spent 15 years at Google, my first job straight out of school. I worked at the intersection of marketing, advertising, and data. Most people would know this work stream as Google Analytics, which helps advertisers and marketers understand the performance of sites, ads, and digital platforms. Naturally, much of my team’s focus at Google centered on those types of problems. Along the way, I became passionate about helping businesses make better decisions using data. That drive pushed me to consider how I could broaden and generalize this work.

This led to early conversations with Databricks a few years ago. At the time, Databricks had the engines and backends to enable powerful solutions, but the UI, customer experience, and product orientation weren’t quite there yet.

A: Your extensive experience and insights in this realm are impressive! You shared that you had early conversations with folks at Databricks before joining. What ultimately drew you to become a Brickster?

While many of the products at the time were built primarily for data scientists and data engineers, we were only beginning to make headway with SQL analysts. There was still a clear gap in business intelligence for business users such as those in finance and marketing. I felt drawn to Databricks because of the opportunity to build out BI solutions and apply AI in ways that could transform business intelligence.

A: I love that you turned that gap in the BI product space into an opportunity, not just for your career, but also for users. What keeps you most excited about the work at Databricks?

In the simplest terms, making our products really useful for customers excites me the most.

Every business has data—or will in the near future. Organizations and individuals alike will want to make sense of that data and use it to drive better decisions.

My goal is to ensure they can use their data as effectively as possible rather than relying solely on gut instinct.

A: Your team is working at the intersection of Gen AI and BI, but what makes this moment so unique in the industry right now?

BI has been going through significant disruption over the last few years. While there were many interesting ideas explored, it wasn’t until GenAI began gaining traction that real opportunities emerged to rethink and innovate beyond traditional methods.

It’s still early days, but that’s what makes it exciting. We now have a real chance to significantly improve workflows and decision-making for more businesses, especially by bringing data and business teams closer together.

Q: When building experiences like Genie, dashboards, or reporting tools, what technical challenges are the most exciting or hardest to solve?

Naturally, there are a lot of challenges. I tend to think of them in three big buckets:

1. User experience: Not purely a technical challenge, but critically important. It’s about designing frictionless experiences so users can accomplish tasks in as little time and with as few clicks as possible.
2. Scalable backends: We need the fastest, most scalable systems to serve thousands, millions, or one day billions of users, delivering answers quickly and reliably over the massive amounts of data that businesses have.
3. Learning systems and feedback loops: The challenge, especially on the GenAI side, is building systems that provide accurate, high-quality suggestions with minimal user effort. While users will always need to give some guidance to teach the systems about their unique business semantics, we want that process to feel as painless as possible.

A: How do you see AI transforming the way businesses interact with BI in the near future?

My hope is that AI continues unlocking a lot more productivity. For the data teams, AI can accelerate many tasks. Hopefully, everything from simple autocompletes to first drafts of analysis can be automated so they can spend more time validating results instead of generating them manually.

Tools like Genie can open new doors for business teams. Instead of waiting for an analyst to respond to a ticket, they can ask the data questions themselves and get answers instantly.

A: Last month, you played a leadership role in opening the new Databricks Vancouver R&D hub. Can you share more about why this expansion is significant?

Three years ago, Databricks acquired a company called Datajoy. That was one of the first acquisitions I worked on in an effort to accelerate our roadmap around BI. The experience was a positive indicator of the strong talent pool in Vancouver, particularly with candidates who bring strong technical expertise and BI domain experience. We’re really excited to double down on this direction by formally opening an R&D hub in Vancouver, and if you’re interested as a candidate, you can view our open roles on our Careers Site here.

A: What kind of talent or culture are you hoping to see there or build there?

We’re looking for talent across all seniorities and all parts of the stack. We're looking to build full-fledged products, so we're looking for folks who can actually join us, are curious about where this can go, and are motivated to actually build across the whole stack. Beyond the technical skills, we look for candidates who embody our culture principles, such as being customer-obsessed, truth-seeking, and collaborative.

A: Exciting times ahead! On leadership and team-building: how do you balance fostering innovation with delivering at scale?

Fostering innovation, while still delivering at scale, involves always keeping the customer in mind. The challenge is balancing the many things we could do with the conviction around the features we should do, those that are most useful and impactful.

It’s not always an easy choice, and it often requires debate, but grounding decisions in customer value helps us strike that balance.

A: Let's move on to talk a little bit about your leadership philosophy. What's it like when you're guiding highly technical teams?

It’s all about giving your teams the right context and making sure they have the right support so that they can focus on the right things. Then stepping back and giving them the space to build rapidly until you see that they need more help and context.

A: What's one lesson you've learned about leading engineers that you wish you knew earlier in your career?

Quite often, when building the first iteration of a product, it's not about whether you should build something to scale now. But whether you know how to build something to scale, and then can intentionally choose to cut half the corners in the right way so that you can get it out faster and validate whether that's the right solution.

A: Where do you see the AI/BI space heading in the next few years?

Hopefully, if we do well, I'd love for us to address the needs of many, many more customers and, within each customer, nearly every employee.

Over time, I'd love to figure out how we actually make it easier to get started. That way, we’d also be able to cater to the larger pool of smaller businesses that don't quite have as much of the data expertise, but are still very eager to make use of it.

A: What advice would you give to engineers who want to build impactful products in this space?

If you have an idea, try out the cheapest version to validate whether it is useful. Then, go from there!

A: Before we close out our conversation, can you share what inspires you outside of work?

I think of recent years, parenting! Watching a small kid grow up has many parallels with trying to train all sorts of interesting AI models, in more ways than I expected.

If you’re interested in joining our teams, visit our Careers Site here.