Skip to main content

- - Databricks for Industries
    Communications
    Media and Entertainment
    Financial Services
    Public Sector
    Healthcare & Life Sciences
    Retail
    Manufacturing
    See All Industries
  - Cross Industry Solutions
    AI Agents
    Cybersecurity
    Marketing
  - Migration & Deployment
    Data Migration
    Professional Services
  - Solution Accelerators
    Explore Accelerators
    Move faster toward outcomes that matter
- - Company
    Who We Are
    Our Team
    Databricks Ventures
    Contact Us
  - Careers
    Working at Databricks
    Open Jobs
  - Press
    Awards and Recognition
    Newsroom
  - Security and Trust
    Security and Trust
JUNE 15–18|SAN FRANCISCO
Don’t miss our biggest Summit yet. Save 50% with early-bird pricing.
Register

Ready to get started?
Get a Demo

JUNE 15–18|SAN FRANCISCO

Don’t miss our biggest Summit yet. Save 50% with early-bird pricing.

Login
Try Databricks

Blog
James Malone

James Malone

Director,Product Management, Databricks

James Malone's posts

Bringing Declarative Pipelines to the Apache Spark™ Open Source Project

Open Source

June 12, 2025/5 min read

Bringing Declarative Pipelines to the Apache Spark™ Open Source Project

Why Databricks

Discover

For Executives
For Startups
Lakehouse Architecture
Databricks AI Research

Customers

Customer Stories

Partners

Partner Overview
Partner Program
Find a Partner
Partner Spotlight
Cloud Providers
Partner Solutions

Discover

For Executives
For Startups
Lakehouse Architecture
Databricks AI Research

Customers

Customer Stories

Partners

Partner Overview
Partner Program
Find a Partner
Partner Spotlight
Cloud Providers
Partner Solutions

Product

Databricks Platform

Platform Overview
Sharing
Governance
Artificial Intelligence
Business Intelligence
Database
Data Management
Data Warehousing
Data Engineering
Data Science
Application Development

Pricing

Pricing Overview
Pricing Calculator

Integrations and Data

Marketplace
IDE Integrations
Partner Connect

Databricks Platform

Platform Overview
Sharing
Governance
Artificial Intelligence
Business Intelligence
Database
Data Management
Data Warehousing
Data Engineering
Data Science
Application Development

Pricing

Pricing Overview
Pricing Calculator

Integrations and Data

Marketplace
IDE Integrations
Partner Connect

Solutions

Databricks For Industries

Communications
Financial Services
Healthcare and Life Sciences
Manufacturing
Media and Entertainment
Public Sector
Retail
View All

Cross Industry Solutions

Cybersecurity
Marketing

Professional Services

Solution Accelerators

Databricks For Industries

Communications
Financial Services
Healthcare and Life Sciences
Manufacturing
Media and Entertainment
Public Sector
Retail
View All

Cross Industry Solutions

Cybersecurity
Marketing

Resources

Customer Support

Learning

Training
Certification
Free Edition
University Alliance
Databricks Academy Login

Events

Data + AI Summit
Data + AI World Tour
AI Days
Event Calendar

Blog and Podcasts

Databricks Blog
Databricks AI Research Blog
Data Brew Podcast
Champions of Data & AI Podcast

Learning

Training
Certification
Free Edition
University Alliance
Databricks Academy Login

Events

Data + AI Summit
Data + AI World Tour
AI Days
Event Calendar

Blog and Podcasts

Databricks Blog
Databricks AI Research Blog
Data Brew Podcast
Champions of Data & AI Podcast

About

Company

Who We Are
Our Team
Databricks Ventures
Contact Us

Careers

Open Jobs
Working at Databricks

Press

Awards and Recognition
Newsroom

Security and Trust

Company

Who We Are
Our Team
Databricks Ventures
Contact Us

Careers

Open Jobs
Working at Databricks

Press

Awards and Recognition
Newsroom

Databricks Inc.
160 Spear Street, 15th Floor
San Francisco, CA 94105
1-866-330-0121

See Careers
at Databricks

© Databricks 2026. All rights reserved. Apache, Apache Spark, Spark, the Spark Logo, Apache Iceberg, Iceberg, and the Apache Iceberg logo are trademarks of the Apache Software Foundation.

Privacy Notice
Terms of Use
Modern Slavery Statement
California Privacy
Your Privacy Choices