Databricks Expands Brickbuilder Program to Include Unity Catalog Accelerators
Today, we're excited to announce the launch of Brickbuilder Unity Catalog Accelerators. This is an expansion to the Brickbuilder Accelerator program , which...
Simplify PySpark testing with DataFrame equality functions
The DataFrame equality test functions were introduced in Apache Spark™ 3.5 and Databricks Runtime 14.2 to simplify PySpark unit testing. The full set...
A Deep Dive into the Latest Performance Improvements of Stateful Pipelines in Apache Spark Structured Streaming
This post is the second part of our two-part series on the latest performance improvements of stateful pipelines. The first part of this...
Performance Improvements for Stateful Pipelines in Apache Spark Structured Streaming
Introduction Apache Spark™ Structured Streaming is a popular open-source stream processing platform that provides scalability and fault tolerance, built on top of the...
Databricks adds new migration Brickbuilder Solutions to help customers succeed with AI
For the past two years, Databricks has collaborated with leading consulting partners to build innovative solutions for industry, migration, and data and AI...
Announcing Ray Autoscaling support on Databricks and Apache Spark™
Ray is an open-source unified compute framework that simplifies scaling AI and Python workloads in a distributed environment. Since we introduced support for...
Parameterized queries with PySpark
PySpark has always provided wonderful SQL and Python APIs for querying data. As of Databricks Runtime 12.1 and Apache Spark 3.4, parameterized queries...
Introducing Mixtral 8x7B with Databricks Model Serving
Today, Databricks is excited to announce support for Mixtral 8x7B in Model Serving . Mixtral 8x7B is a sparse Mixture of Experts (MoE)...
Offline LLM Evaluation: Step-by-Step GenAI Application Assessment on Databricks
Background In an era where Retrieval-Augmented Generation (RAG) is revolutionizing the way we interact with AI-driven applications, ensuring the efficiency and effectiveness of...
Lakehouse Monitoring: A Unified Solution for Quality of Data and AI
Introduction Databricks Lakehouse Monitoring allows you to monitor all your data pipelines – from data to features to ML models – without additional...