From Warehouse to Lakehouse: Migration Approaches to Databricks

Before making architectural decisions, it’s worth revisiting the broader migration strategy. In our previous post, we introduced Databricks Professional Services’ approach to complex data warehouse migrations, highlighting the importance of early decisions around strategy and design. These foundational choices directly influence the target platform's implementation and architecture.

We also introduced two sequencing strategies: ETL-first and BI-first. The BI-first approach delivers quick value by modernizing the consumption layer, while the ETL-first approach focuses on upstream pipelines. Each has its place, depending on priorities.

In this post, we explore one of the most critical design choices: choosing between a Lift-and-Shift or Modernization approach. We explain what each approach entails, when to use it, and how to merge them into a hybrid approach for long-term success on Databricks.

From strategy to migration approach: picking the right path

After you’ve aligned on the broader migration strategy—ETL-first or BI-first—the next major decision is how to structure the migration. Do you replicate what exists, or reimagine it for the future?

This architectural decision typically comes down to two core approaches:

• Lift-and-Shift: Move workloads as-is to accelerate the migration
• Modernization: Redesign the platform to unlock long-term efficiency and scale

The right approach depends on your goals, constraints, and timeline. Below, we break down the tradeoffs of each and include a hybrid model that many organizations use to combine the best of both.

Lift and shift migration 

Lift-and-Shift involves moving your existing data models and codebase to the new platform with minimal changes. You don’t introduce new use cases, and the architecture remains unchanged.

This approach is appealing because it’s easier to scope, plan, and automate. Tools like profilers and code analyzers help measure workload patterns, complexity, and cost, making it easier to evaluate and execute.

Key benefits include:

• Predictable timelines
• Automated tooling (e.g., code converters, reconciliation validators)
• Faster migration when facing deadlines or expiring licenses

For example, code converters can automatically handle up to 80% of scripts. Since functionality remains the same, validation and running queries on both systems and comparing outputs are easier.

On Databricks, Lift-and-Shift gets you off legacy platforms quickly while unlocking immediate performance gains using features like z-ordering and liquid clustering. After you have migrated, your organization can begin incrementally modernizing the platform.

Modernize the migration pattern

Modernizing, in contrast to Lift-and-Shift, means building a new data platform in your target system without being constrained by your legacy architecture. The focus shifts from simply migrating existing assets to reimagining use cases and designing for future needs. Instead of mapping outdated optimizations, you implement best practices and the well-architected pillars of the lakehouse.

On an open lakehouse, this involves refactoring code and re-architecting data structures to meet your organization’s current and future scalability, performance, cost, and capability requirements, free from legacy limitations.

Tooling remains useful, but more for discovery and planning:

• Profilers and code analyzers help inventory what you need to migrate
• Code converters and reconciliation tools play a minimal role, since this isn’t a direct code translation

This approach is ideal when you have flexible timelines and an outdated or overly complex legacy system, often with thousands of tables and scripts. While starting fresh can feel slow and overwhelming, the long-term benefits are substantial: simplified architecture, better performance, and reduced maintenance overhead.

That said, migrating thousands of scripts often means maintaining their maintenance complexity. If that seems daunting, consider partnering with Databricks Professional Services or certified migration experts to help guide the planning and design phase and ensure a smoother path.

A hybrid approach: lift and shift, and then modernize

Another approach is a hybrid migration strategy that balances speed with long-term value. You would begin with the Lift-and-Shift approach to eliminating their legacy platform as quickly as possible, especially when facing urgent constraints like expiring licenses. Automation and repeatable tooling help accelerate this initial phase and reduce risk during execution.

You can move into the modernization phase after you migrate your workloads to Databricks. 

In the hybrid approach, you:

• Integrate new and modern data sources
• Implement a data product strategy
• Enable advanced analytics, AI, and new use cases that drive business decisions

This phase often requires architectural updates but allows you to evolve gradually. With a hybrid strategy, you don’t have to modernize everything on day one—you build on a stable foundation while aligning with future requirements.

If you're pursuing this approach, Databricks Professional Services and certified partners can help guide your roadmap, ensuring a smooth transition and a future-ready architecture.

Our point of view

Selecting a migration approach is not a one-size-fits-all. The most common approach is a hybrid migration:

1. Create a migration factory that leverages automation tools.
2. Lift-and-shift the majority of the codebase. 
3. Enable out-of-the-box optimizations, such as z-ordering and liquid clustering, to start your modernization effort.

Databricks can act as your primary data warehouse. For example, you can migrate stored procedures to notebooks and use SQL Scripting for scalability and AI integration without leaving the comfort of SQL. Migrating Transact-SQL to any other cloud data warehouse requires a similar effort to migrating that Transact-SQL to a notebook with Python code wrapped around your SQL functionality. The benefit of using a notebook is that you also get flexibility and a great development experience.

What to do next

Ready to modernize your data warehouse? Download our eBook, “Transforming Legacy Data Warehouses: A Strategic Migration Blueprint,” for detailed strategies and best practices that ensure a low-risk transition to the Databricks Data Intelligence Platform.