Building Query Expert MCP: Using RAG to Build an Analytics Agent That Goes Beyond Text-to-SQL

LLMs are fast and confident SQL writers that can unlock self-serve analytics across your business. But actionable insights require more than SQL syntax—they demand business context, domain expertise and nuances that live outside your schema docs.

In this talk, we'll deep-dive into how Block built Query Expert MCP, a scalable AI agent that captures and distributes institutional data knowledge using Databricks. You'll learn how we:

• Automated documentation at scale—using query_ai sub-agents to generate standardized table documentation across 11,000+ tables
• Captured SQL nuances—Databricks sub-agents categorize and label domain-specific query patterns across 1M sample queries
• Human in the loop—Databricks Apps allow users to directly edit, remove or upload context to the agent’s memory
• Enabled intelligent retrieval—Databricks Vector Search indexes for RAG-powered query generation that scales to a 12,000 person org, while supporting domain-specific scoping for teams' use cases