How Databricks Ships AI Safely: Inside the Team Enabling Secure Innovation
Overview
| Experience | In Person |
|---|---|
| Track | Cybersecurity |
| Industry | Enterprise Technology, Healthcare & Life Sciences, Financial Services |
| Technologies | AI/BI, Unity Catalog, Databricks Agents |
| Skill Level | Intermediate |
| DOWNLOAD SESSION SLIDES | |
The pace of AI development has reset what security teams need to deliver. At Databricks, we've built an AI Security team that operates at the frontier of the technology it defends. This session is an inside look at how.
We'll share the human-in-the-loop agents we built to assess security issues across our AI products and services, and why an early, disciplined bet on agent benchmarking got us to 95.8% accuracy on agent-generated findings, and how we use expert reviewers validating every result so nothing slips through.
Then we'll open the hood on how we secure the coding agents our own engineers use every day: managed client-side settings, agent sandboxes, an AI Gateway and MCP Gateway enforcing policy server-side where admin access can't reach, and continuous scanning of every Databricks App, plugin, and skill.Real architecture. Real numbers. Real failure modes. A practical look at what a modern AI security function looks like in production.
Session Speakers
Alex Warnecke
/AI Security Engineer
Databricks
Nishith Sinha
/Senior Engineering Manager- AI Security
Databricks, Inc.
Full Summary
Inside the Databricks AI security playbook: agents reviewing agents
Generative AI has upended software delivery speed and risk. The conversation examines how Databricks scales AI security reviews with an agentic reviewer, while hardening the coding agents employees use day to day. The focus is on practical controls and evaluation, not silver bullets, reflecting the operational tradeoffs behind security best practices on the Databricks Platform. Teams building on the platform can also consult the documented security features and best practices for a structured starting point when designing controls around sensitive data and agent access.
FAQ
It is the combination of sensitive data access, untrusted inputs, and external communication. When all three are present, prompt injection can trigger harmful actions like data exfiltration. Reducing any one leg limits blast radius, but in AI platforms all three are often unavoidable, so detection and layered controls are essential.
Directional agreement with prior human reviews is about 95.8 percent across roughly 600 question-answer pairs. It is strongest on questions with clear code signals, such as LLM usage, authentication, and logging. It is weaker on judgment-heavy calls, for example whether system prompt content is sensitive in context.
Residual disagreements cluster around business and contextual judgment, and the agent can hallucinate issues like nonexistent API keys. Humans own the verdict, especially for higher-risk features, which keeps accountability clear and reduces the chance of missed consequences.
Engineers often have admin rights and could override local settings, or attackers could tamper with them on a compromised device. Server-side AI and MCP gateways backstop clients by enforcing human approval for mutating actions, masking PII, steering model behavior, and failing closed when approvals or services are unavailable.
Skills, plugins, and internal apps are continuously scanned using regex, YARA signatures, and LLM-as-judge. Suspicious behaviors, such as unexpected external endpoints, trigger incident response and host quarantine, which limits blast radius while the item is investigated.