Session
Chaos to Clarity: Secure, Scalable, and Governed SaaS Ingestion through Lakeflow Connect and more
Overview
Experience | In Person |
---|---|
Type | Breakout |
Track | Data Engineering and Streaming |
Industry | Enterprise Technology, Professional Services |
Technologies | Apache Spark, LakeFlow, Unity Catalog |
Skill Level | Beginner |
Duration | 40 min |
Ingesting data from SaaS systems sounds straightforward—until you hit API limits, miss SLAs, or accidentally ingest PII. Sound familiar? In this talk, we’ll share how Databricks evolved from scrappy ingestion scripts to a unified, secure, and scalable ingestion platform. Along the way, we’ll highlight the hard lessons, the surprising pitfalls, and the tools that helped us level up. Whether you’re just starting to wrangle third-party data or looking to scale while handling governance and compliance, this session will help you think beyond pipelines and toward platform thinking.
Session Speakers
Prashant Gupta
/Software Engineer
Databricks
Krishna Bhupatiraju
/Sr. Engineering Manager
Databricks