Introducing Lakehouse Apps

Lakehouse Apps is a new way to build native applications for Databricks. Lakehouse Apps will offer the most secure way to build, distribute, and run innovative data and AI applications directly on the Databricks Lakehouse Platform, next to the customer’s data, with the full security and governance capabilities of Databricks. 

For developers building data and AI solutions, distributing Lakehouse Apps through the Databricks Marketplace will dramatically reduce time to adoption, with access to over 10,000 Databricks customers. For customers, Lakehouse Apps will be the most secure way to run applications that unlock the full value of data in their Lakehouse, leverage Databricks-native services, and extend Databricks with new capabilities.

The Data Application Dilemma

Data and AI applications are among the fastest-growing software categories, and the growth in generative AI and large language models (LLMs) has accelerated that trend. Developers are developing innovative data and AI solutions every day, and development teams in companies are building in-house applications that directly use data in a lakehouse, or LLMs and other AI models built and trained on that data.

But getting these solutions into the hands of users is unnecessarily complex. 

• A company’s data is its most valuable asset and is often subject to strict security and regulatory requirements. Giving external applications access to that data usually requires clearing difficult legal, security, and commercial hurdles. That process is expensive and can sometimes even block a software vendor from moving forward with a customer.
• Data and AI workloads require significant compute capacity and scale. Building and operating the services needed to deliver these applications reliably and scalably across multiple clouds and regions takes a lot of effort. 
• Enterprise-ready applications need to be integrated with a customer’s security, management, and governance systems. These integrations are costly to build, and new applications often lack them.  

To address these challenges, developers often resort to one of two alternatives:

• Compromise: Developers can rewrite parts of their application in SQL or proprietary scripts and frameworks to run safely within a customer’s data platform. This makes it hard to deliver the full functionality of the application, and requires writing and testing complex code for each platform.
• Complexity: Developers can build and distribute “on-premises” versions of their products, and require customers to manually install, configure, secure, and operate them in their own VPC. This process is time-consuming, unreliable, and doesn’t scale.

We can do better. 

Lakehouse Apps: Security Without Compromise

Lakehouse Apps addresses these challenges with a native, secure, no-compromise solution.

Lakehouse Apps run directly on a customer’s Databricks instance, where they can integrate with the customer’s data, use and extend Databricks services, and enable users to interact with a single sign-on experience - all with the same security, privacy, and compliance controls as Databricks. Data never needs to leave the customer’s instance. 

Lakehouse Apps can be instantly distributed and monetized to over 10,000 Databricks customers through the Databricks Marketplace. Customers can discover, install, secure, manage, and govern them as efficiently as Databricks native features. Compute resources used by apps are billed directly to the customer by Databricks. A customer’s internal apps can also be directly installed into an instance.

Lakehouse Apps are secured, sandboxed, and governed. Every app runs within a secure, configurable sandbox. Customers can use Unity Catalog, Databricks’ unified governance solution, to select which resources an app can access, control who can interact with the app, and automatically govern an app’s activities. 

Lakehouse Apps are built with the technology of your choice. Apps run on secure, auto-scale compute that runs containerized code that can be written in virtually any language, so developers are not limited to building in any specific framework. Applications that integrate with Databricks today can be easily converted into Apps. 

Lakehouse Apps are fully Lakehouse-native. The Databricks platform includes a powerful set of scalable and cost-effective serverless services, including fast data warehousing, workflows and pipelines, and AI/LLM training and serving. Lakehouse applications can leverage any of these capabilities. Apps can also use catalogs and contribute metadata and lineage in Unity Catalog, integrate with the Lakehouse filesystem, and extend the lakehouse with custom functionality.

Benefits of Lakehouse Apps

For developers: 

• Reach over 10K customers instantly with your data and AI solutions.
• Accelerate time to adoption dramatically.
• Reuse your existing investments by leveraging the platform and code of your choice.
• Leverage the Databricks Lakehouse Platform to scale your applications and services.
• Extend Databricks to reach hundreds of thousands of data and AI practitioners. 
• Benefit from being part of Databricks ecosystem and programs such as Databricks for Startups.

For customers:

• Give your users safe and easy access to a wide range of innovative new applications.
• Build and distribute in-house data applications as easily and securely as notebooks and dashboards. 
• Reduce time and effort to adopt, integrate, and manage data and AI applications. 

What Partners Are Saying

We’re looking forward to bringing Lakehouse Apps to Databricks in the near future. We are now working with selected early partners, including Retool, Posit, Plotly, Immuta, Kumo.ai, Lamini, Codeium and Habu, to build their innovative solutions as Lakehouse Apps. 

"Developers use Retool to build and deploy custom business software with an easy-to-use graphical user interface that sits on top of any data stack. Retool’s native integration with Databricks allows builders to create applications that get information into the hands of those that need it most — without compromising on speed, security, or reliability. Data engineers, data scientists, software developers and end users of data applications can now all benefit from our powerful offering listed on the Databricks Marketplace."

– Mark Schaaf, COO, Retool

"In a partnership with Databricks, we are excited to offer R and Python users seamless access to their favorite authoring environments, such as RStudio, Jupyter, and VS Code. By bringing Posit Workbench to the Databricks Marketplace, we significantly lower the barrier for our joint customers, granting them secure data access while harnessing the security, reliability, scalability, and performance of the Databricks Lakehouse Platform. As we move forward, we eagerly anticipate supporting over 10,000 Databricks customers with the advanced publishing capabilities that our loyal Posit Connect users currently enjoy." 

– Tareef Kawaf, President of Posit Software, PBC

"Immuta's Data Security Platform enables organizations to unlock value from their cloud data. With Lakehouse Apps, we can bring our offering closer to the data managed in Databricks and enable our joint customers to get started faster. With the flexibility of Databricks Apps, customers can reuse existing code and skills, and build a closer integration with the Lakehouse platform."

– Mo Plassnig, Chief Product Officer, Immuta

"Plotly’s Dash Enterprise is the leading platform for building production-grade data applications. With 3200 common organizations using Plotly & Dash and Databricks, Plotly is a major Databricks data and AI partner. We are excited to be working with Databricks to ensure that our users can work with their data in an even more tightly integrated manner, including data authentication and authorization, while benefiting from the security, governance, and scalability of the Databricks Lakehouse platform."

– Chris Parmer, Co-Founder of Plotly and Creator of Dash

Are you interested in being a Lakehouse Apps partner? Contact Us and sign up for updates!