Building Query Expert MCP: Using RAG to Build an Analytics Agent That Goes Beyond Text-to-SQL

LLMs are fast and confident SQL writers that can unlock self-serve analytics across your business. But actionable insights require more than SQL syntax—they demand business context, domain expertise and nuances that live outside your schema docs.In this talk, we'll deep-dive into how Block built Query Expert MCP, a scalable AI agent that captures and distributes institutional data knowledge using Databricks. You'll learn how we:

• Automated documentation at scale—using query_ai sub-agents to generate standardized table documentation across 11,000+ tables
• Captured SQL nuances—Databricks sub-agents categorize and label domain-specific query patterns across 1M sample queries
• Human in the loop—Databricks Apps allow users to directly edit, remove or upload context to the agent’s memory
• Enabled intelligent retrieval—Databricks Vector Search indexes for RAG-powered query generation that scales to a 12,000 person org, while supporting domain-specific scoping for teams' use cases