Session

Enterprise AI at Scale: Serving Google Gemini on Databricks for Production Workloads

Overview

ExperienceIn Person
TrackArtificial Intelligence & Agents
IndustryHealthcare & Life Sciences, Manufacturing, Financial Services
TechnologiesDatabricks SQL, Unity Catalog, Lakebase
Skill LevelAdvanced
DOWNLOAD SESSION SLIDES

Foundation models are moving beyond chatbots and copilots into the core of enterprise data pipelines — enriching records, classifying documents, resolving entities, and powering agentic workflows at massive scale. This session shows how Google Gemini and Databricks are partnering to make this real for production enterprises today.

 

Google Cloud shows how Gemini is available as a first-party model on Databricks, served natively via ai_query() and Model Serving — with built-in governance via AI Gateway, including cost controls, rate limits, usage tracking, and Unity Catalog logging.

 

HG Insights then brings this to life with a production use case: a multi-stage AI pipeline processing 7 billion records where rules and custom BERT models handle the predictable cases, and Gemini acts as an agentic enrichment layer — autonomously reasoning over incomplete company profiles and resolving missing attributes across hundreds of millions of inference calls. Orchestrated by Lakeflow and monitored via AI/BI dashboards, this pipeline directly generates the revenue-critical datasets HG Insights delivers to Fortune 500 customers.

 

Whether you're enriching CRM data, building document classification pipelines, or deploying agentic data quality workflows, this session gives you the architecture patterns and lessons learned to run Gemini on Databricks at enterprise scale.

Session Speakers

Speaker placeholderIMAGE COMING SOON

Abhishek Bhagwat

/Software Engineer, Cloud AI
Google Cloud

Bhadri Pani

/Senior Machine Learning Engineer
HG Insights

Speaker placeholderIMAGE COMING SOON

Shailesh Dargude

/AI Fellow
HG Insights

Full Summary

Running Gemini on Databricks: how HG Insights scales agentic pipelines across billions of documents

A joint Google Cloud and HG Insights session shows how to operationalize Gemini models inside the Databricks Platform, moving from single-SQL inference to managed agents that can query enterprise data on demand. The discussion centers on what it actually takes to run agentic AI in production, where cost, governance, and evaluation determine whether systems scale beyond prototypes. For broader context on how the analyst role is changing, Databricks has written about why AI is rewriting the analyst's job description, and teams building toward fully autonomous workflows can also explore what agentic analytics means in practice.

FAQ


Use Flash for high-volume agentic work where speed and cost matter, including row-by-row inference across large tables. Switch to Pro when you need maximum reasoning or more complex, multi-step coding patterns. In most production data workflows at scale, Flash is the default.

AI_QUERY lets you apply a prompt to every row of a Databricks table with a single SQL call. The platform manages batching, checkpointing, scaling, and rate limits, which removes most of the orchestration code teams would otherwise write to handle API quotas and failure recovery across millions of requests.

The agent registry stores an immutable agent configuration, including instructions, tools, and auth, created via a one-time control-plane call. Centralizing definitions improves governance, prevents configuration drift, and ensures updates propagate consistently rather than diverging across runtime instances.

Running frontier models over billions of documents is prohibitively expensive and discards hard-won domain knowledge. A layered approach uses deterministic filters and small fine-tuned Gemma models first, escalating only ambiguous cases to Gemini Flash, with human-in-the-loop for novel products.

The gateway supports usage tracking, inference tables for logging, rate limits for fair usage, and configurable guardrails. Those controls apply uniformly to model endpoints that downstream SQL or agent workloads consume.