How Gemini Built a Cryptocurrency Analytics Platform Using Lakehouse for Financial Services
This blog has been co-authored by Gemini. We would like to thank the Gemini team, Anil Kovvuri and Sriram Rajappa, for their contributions...
Deploy Production Pipelines Even Easier With Python Wheel Tasks
With its rich open source ecosystem and approachable syntax, Python has become the main programming language for data engineering and machine learning. Data...
Databricks Delta Live Tables Announces Support for Simplified Change Data Capture
As organizations adopt the data lakehouse architecture, data engineers are looking for efficient ways to capture continually arriving data. Even with the right...
Saving Time and Cost With Cluster Reuse in Databricks Jobs
With our launch of Jobs Orchestration , orchestrating pipelines in Databricks has become significantly easier. The ability to separate ETL or ML pipelines...
OMB M-21-31: A Cost-Effective Alternative to Meeting and Exceeding Traditional SIEMs With Databricks
On August 29, 2021, the U.S. Office of Management and Budget (OMB) released a memo in accordance with the Biden Administration’s Executive Order...
Google Datastream Integration With Delta Lake for Change Data Capture
This is a collaborative post between the data teams as Badal, Google and Databricks. We thank Eugene Miretsky, Partner, and Steven Deutscher-Kobayashi, Senior...
Streamline MLOps With MLflow Model Registry Webhooks
As machine learning becomes more widely adopted, businesses need to deploy models at speed and scale to achieve maximum value. Today, we are...
Orchestrating Databricks Workloads on AWS With Managed Workflows for Apache Airflow
In this blog, we explore how to leverage Databricks’ powerful jobs API with Amazon Managed Apache Airflow (MWAA) and integrate with Cloudwatch to...
Building Data Applications on the Lakehouse With the Databricks SQL Connector for Python
We are excited to announce General Availability of the Databricks SQL Connector for Python . This follows the recent General Availability of Databricks...
Hunting Anomalous Connections and Infrastructure With TLS Certificates
According to Sophos, 46% of all malware now uses Transport Layer Security (TLS) to conceal its communication channels. A number that has doubled...