Databricks and AWS Partner to Simplify Infrastructure Setup

Today, we’re announcing that automated configuration for classic workspace deployment on AWS is Generally Available. Customers can now deploy new workspaces in minutes, directly from the Databricks account console. Powered by AWS IAM temporary delegation, this new flow automates infrastructure setup, allowing customers to focus more of their time on building data intelligence.

Databricks on AWS is rapidly growing at an unprecedented rate. As this growth accelerates, our customers have made it clear that they want to focus on data intelligence, not cloud administration.

Previously, setting up a Databricks workspace on AWS required cloud admins to manually configure infrastructure across both Databricks and AWS, taking up to an hour.

Databricks and AWS have partnered to simplify customer onboarding at every step. Last year at AWS re:Invent, we announced SaaS Quick Launch for streamlined AWS Marketplace onboarding and Buy with AWS for faster procurement.

In this blog, we'll cover:

• How automated configuration works with AWS IAM temporary delegation
• How to set up a workspace with the new flow

How automated configuration works

To deploy classic Databricks workspaces, customers need to provide their AWS credentials to provision and manage EC2 compute and S3 storage resources. Previously, cloud admins manually created IAM roles with 140+ line JSON policies, configured S3 buckets, and set up VPC networking, a process that took up to an hour.

Now, we’ve made it a few simple clicks:

What’s happening here?

Powered by AWS IAM temporary delegation, Databricks automatically provisions all required resources when users select “Add automatically” during workspace creation:

Compute credentials:

• Cross-account IAM role with scoped permissions and proper trust policies for classic compute access and lifecycle management
• Customer-managed VPC with default subnets, security groups, and routing tables

Storage credentials:

• S3 bucket with properly configured access policies
• Separate IAM role that grants Unity Catalog least-privilege access to the S3 bucket
• Managed file events are automatically enabled for efficient data processing

Delegated permission check:

After logging into AWS, the temporary delegation integration verifies the user’s AWS permissions against the required ones for creating a classic workspace.

• If they have sufficient access, users grant Databricks temporary permissions (for a specified duration) to automatically provision all necessary AWS resources. If not, they can request the required permissions from their AWS account admin within the same flow.
• All delegated permissions are time-bounded and automatically expire after deployment, reducing standing access and security risk. Customers always review and approve requested permissions in the AWS console before any resources are created.

Key benefits of automated configuration

• Eliminates common errors: Automated provisioning prevents mistakes like incorrect trust policies, missing S3 permissions, or misconfigured IDs
• Built-in approval workflows: Users without the necessary permissions can request them from their AWS account admins, eliminating the most common classic workspace creation failure
• Least-privilege security by default: All IAM roles follow least-privilege principles with scoped permissions and boundaries that align with enterprise security policies
• Managed file events enabled: Automatically configured for efficient data processing with Autoloader
• Customer-managed VPCs by default: Every workspace deploys in a customer-managed VPC, with the option to add enterprise security features post-deployment
• Complete audit trail: All automated actions are logged in AWS CloudTrail with full visibility into created resources

Learn More

If you want to create a new classic workspace using new credentials, navigate to your Databricks account console and try the new automated configuration deployment experience today. Click “Create Workspace” from the Workspace tab to get started.

If you are new to Databricks, sign up for our trial.

For more information:

• Learn more about workspace deployment options
• Additional information on AWS IAM temporary delegation

Join us at AWS Reinvent, Dec 1-5 at the Venetian in Las Vegas! Learn how leading organizations are accelerating their data modernization journeys on AWS and get hands-on experience with the latest innovations.