Building Permission-Aware AI for Enterprise Content with LakeFlow Connect
Overview
| Experience | In Person |
|---|---|
| Track | Data Engineering & Streaming |
| Industry | Enterprise Technology, Retail & Consumer Goods, Financial Services |
| Technologies | Lakeflow, Agent Bricks |
| Skill Level | Intermediate |
In this session, we’ll explore how to build scalable, permission-aware AI applications by ingesting files and metadata from Google Drive and SharePoint into Databricks.We’ll examine how LakeFlow Connect ingests structured files (Excel, JSON), semi-structured data, and unstructured documents (PDF, DOCX, and more) from enterprise content systems, while capturing rich metadata and preserving file-level permissions for secure downstream use. You’ll learn how to process change feeds, handle deletes, manage schema evolution, and synchronize access controls, ensuring that AI systems respect enterprise security boundaries without sacrificing performance or scalability.
We’ll then showcase how this ingestion foundation powers high-impact GenAI applications, including knowledge assistants and enterprise search with retrieval-augmented generation (RAG), transforming distributed enterprise content into governed, AI-ready intelligence that delivers trusted answers and actionable insights.
Session Speakers
Sue Ann Hong
/Software Engineer
Databricks
Sandip Agarwala
/Staff Software Engineer
Databricks