Session

Building With Community Connectors: Custom Data Sources, Managed Reliability

Overview

ExperienceIn Person
TrackData Engineering & Streaming
IndustryEnterprise Technology
TechnologiesLakeflow
Skill LevelIntermediate

Are you tired of building fragile, costly API scripts for niche data sources that lack enterprise-grade features like lineage, retries and unified governance?Community Connectors are built on the foundational power of Lakeflow Connect and bridge the gap between "DIY" and “fully managed." These new AI-powered, open-source templates help you build, manage and host your own ingestion paths while inheriting many of the robust, built-in features of the Databricks Platform.In this lightning talk, we’ll explore:

  • Built-in tooling: Use out-of-the-box observability, SCD Type 2 handling, automatic retries and more
  • Flexible customization: Use open-source templates to meet requirements that go beyond standard managed connectors
  • Unified governance: Centrally manage credentials, pipelines, data and lineage via Unity Catalog

Join us to get started and learn how they offer the flexibility of a DIY approach with the scale and reliability of a managed Databricks service.

Session Speakers

Peter Pogorski

/Staff Product Manager
Databricks