Skip to main content

Databricks Simplifies the Path to Building Lakehouses for Business Intelligence and Machine Learning

With Databricks Ingest and new partner integrations, data teams can easily populate lakehouses, which combine the best of data lakes and data warehouses

February 24, 2020
Share this post

SAN FRANCISCO - Feb. 24, 2020 - Databricks, the leader in unified data analytics, today announced an accelerated path for data teams to unify data management, business intelligence (BI) and machine learning (ML) on one platform. The new Data Ingestion Network of partners and Databricks Ingest bring data teams closer to building the new data management paradigm, What is a Lakehouse?, which combines the best elements of data lakes and data warehouses, enabling BI and ML on all of a business’s data.

Historically, companies have been forced to split up their data into traditional structured data and big data, and use them separately for BI and ML use cases. This results in siloed data in data lakes and data warehouses, slow processing, and partial results that are too delayed or too incomplete to be effectively utilized. Customers can now load data into Delta Lake, the open source technology for building reliable and fast lakehouses at scale, through the Data Ingestion Network of partners - Fivetran, Qlik, Infoworks, StreamSets, Syncsort - with built-in integrations to Databricks Ingest for automated data loading. Azure Databricks customers already benefit from native integration with Azure Data Factory to ingest data from many sources.

“Databricks powers our machine learning and business intelligence across multiple business functions, from car inventory management, to price prediction and technical operations, by using hundreds of terabytes of data,” said Greg Rokita, executive director, Technology at Edmunds. “Our data vision is fully aligned with the lakehouse approach, and our cloud data journey starts with Delta Lake which powers our machine learning use cases and executive reporting. We’re excited about Databricks Ingest - it will definitely simplify loading data into our Delta Lake.”

Data teams can load data from various sources - applications like Salesforce, Marketo, Zendesk, SAP, and Google Analytics; databases like Cassandra, Oracle, MySQL, and MongoDB and file storage like Amazon S3, Azure Data Lake Storage, Google Cloud Storage into Delta Lake for all their BI and ML use cases. In addition to the integration network partnerships announced today, Informatica, Segment and Talend integrations will soon be available in an upcoming release.

“The Lakehouse paradigm aspires to combine the reliability of data warehouses with the scale of data lakes to support every kind of use case. In order for this architecture to work well, it needs to be easy for every type of data to be pulled in. Databricks Ingest is an important step in making that possible,” says Ali Ghodsi, co-founder and CEO of Databricks.

Additionally, the auto-loading capabilities allow data to continuously flow into Delta Lake, without setting up and maintaining job triggers or schedules. As companies’ data appears in cloud storage from different sources, Databricks Ingest automatically pulls this new data efficiently into Delta Lake. This breaks down the silos so data can be used by teams across a company to deliver data-driven innovation and business value with data science, ML and business analytics.

“Fivetran and Databricks allow customers to bring together big data and business context in a single environment. The combined technology stack enables users to perform both cutting-edge machine learning workloads, and traditional business intelligence, in a single unified lakehouse,” said George Fraser, CEO of Fivetran.

“Qlik is the leader in automated and real-time data integration with cloud data warehouses and data lakes, having moved data from more than 200,000 databases with our unique change data capture (CDC) technology for some of the world’s largest enterprises. We are excited that customers will benefit from Qlik’s optimized data delivery for Delta Lake. Databricks’ users now have a more seamless on-ramp to easily unlock and stream data from all of their enterprise sources including mainframes, SAP, databases and data warehouses, by implementing open lakehouses on top of Delta Lake,” said Mike Capone, CEO of Qlik.

To learn more, read “What is a Lakehouse?” from industry luminaries Ben Lorica, Michael Armbrust, Ali Ghodsi, Reynold Xin and Matei Zaharia, explaining the emergence of this new data management paradigm as the successor to data warehouses and data lakes.

Additional Resources

Data Ingestion Network Blogs

About Databricks
Databricks helps data teams solve the world’s toughest problems. As the leader in Unified Data Analytics, Databricks helps organizations make all their data ready for analytics, empower data-driven decisions across the organization, and rapidly adopt machine learning to outpace the competition. The company’s global customer base has thousands of organizations including Comcast, Shell, Expedia, and Regeneron. Databricks is venture-backed and founded by the original creators of popular open source projects, including Apache Spark, Delta Lake and MLflow. To learn more, follow Databricks on Twitter, LinkedIn and Facebook.
Apache, Apache Spark and Spark are trademarks of the Apache Software Foundation.
Copyright © 2019 Delta Lake, a Series of LF Projects, LLC. For website terms of use, trademark policy and other project policies please see https://lfprojects.org.

Media Contact
Kristalle Cooks
Head of Communications
415-462-4907
[email protected]

Recent Press Releases

Databricks Completes the Financial Security Institute’s Security and Safety Assessment for Cloud Service Providers in Korea
Read Now
Databricks Announces Over 70% Annualized Growth in France as Demand for the Data Intelligence Platform Grows
Read Now
Databricks Achieves Authorization for DoD IL5 on AWS GovCloud
Read Now
New Economist Impact Study Finds Only 22% of Enterprises Believe Their IT Infrastructure is Ready for AI
Read Now
KT Corporation Collaborates with Databricks to Accelerate Data and AI Innovation in Korea
Read Now
View All