Skip to main content

- - Databricks for Industries
    Communications
    Media and Entertainment
    Financial Services
    Public Sector
    Healthcare & Life Sciences
    Retail
    Manufacturing
    See All Industries
  - Cross Industry Solutions
    AI Agents
    AI Governance
    Cybersecurity
    Marketing
  - Migration & Deployment
    Data Migration
    Professional Services
  - Solution Accelerators
    Explore Accelerators
    Move faster toward outcomes that matter
- - Company
    Who We Are
    Our Team
    Databricks Ventures
    Contact Us
  - Careers
    Working at Databricks
    Open Jobs
  - Press
    Awards and Recognition
    Newsroom
  - Security and Trust
    Security and Trust
JUNE 15–18|SAN FRANCISCO
Join us at the world’s largest data, apps and AI event.
Register

Ready to get started?
Get a Demo

JUNE 15–18|SAN FRANCISCO

Join us at the world’s largest data, apps and AI event.

Login
Try Databricks

Blog
Michael Allman

Michael Allman

Michael Allman's posts

Data Intelligence Platforms

Open Source

December 15, 2016/8 min read

Scalable Partition Handling for Cloud-Native Architecture in Apache Spark 2.1

Why Databricks

Discover

For App Developers
For Executives
For Startups
Lakehouse Architecture
Databricks AI Research

Customers

Customer Stories

Partners

Partner Overview
Partner Program
Find a Partner
Partner Spotlight
Cloud Providers
Partner Solutions

Discover

For App Developers
For Executives
For Startups
Lakehouse Architecture
Databricks AI Research

Customers

Customer Stories

Partners

Partner Overview
Partner Program
Find a Partner
Partner Spotlight
Cloud Providers
Partner Solutions

Product

Databricks Platform

Platform Overview
Sharing
Governance
Artificial Intelligence
Business Intelligence
Database
Data Management
Data Warehousing
Data Engineering
Data Science
Application Development
Security

Pricing

Pricing Overview
Pricing Calculator

Integrations and Data

Marketplace
IDE Integrations
Partner Connect

Databricks Platform

Platform Overview
Sharing
Governance
Artificial Intelligence
Business Intelligence
Database
Data Management
Data Warehousing
Data Engineering
Data Science
Application Development
Security

Pricing

Pricing Overview
Pricing Calculator

Integrations and Data

Marketplace
IDE Integrations
Partner Connect

Solutions

Databricks For Industries

Communications
Financial Services
Healthcare and Life Sciences
Manufacturing
Media and Entertainment
Public Sector
Retail
View All

Cross Industry Solutions

AI Agents
AI Governance
Cybersecurity
Marketing

Professional Services

Solution Accelerators

Databricks For Industries

Communications
Financial Services
Healthcare and Life Sciences
Manufacturing
Media and Entertainment
Public Sector
Retail
View All

Cross Industry Solutions

AI Agents
AI Governance
Cybersecurity
Marketing

Resources

Customer Support

Learning

Training
Certification
Free Edition
University Alliance
Databricks Academy Login

Events

Data + AI Summit
Data + AI World Tour
AI Days
Event Calendar

Blog and Podcasts

Databricks Blog
AI Blog
Data Brew Podcast
Champions of Data & AI Podcast

Learning

Training
Certification
Free Edition
University Alliance
Databricks Academy Login

Events

Data + AI Summit
Data + AI World Tour
AI Days
Event Calendar

Blog and Podcasts

Databricks Blog
AI Blog
Data Brew Podcast
Champions of Data & AI Podcast

About

Company

Who We Are
Our Team
Databricks Ventures
Contact Us

Careers

Open Jobs
Working at Databricks

Press

Awards and Recognition
Newsroom

Security and Trust

Company

Who We Are
Our Team
Databricks Ventures
Contact Us

Careers

Open Jobs
Working at Databricks

Press

Awards and Recognition
Newsroom

Databricks Inc.
160 Spear Street, 15th Floor
San Francisco, CA 94105
1-866-330-0121

See Careers
at Databricks

© Databricks 2026. All rights reserved. Apache, Apache Spark, Spark, the Spark Logo, Apache Iceberg, Iceberg, and the Apache Iceberg logo are trademarks of the Apache Software Foundation.

Privacy Notice
Terms of Use
Modern Slavery Statement
California Privacy
Your Privacy Choices