Named Arguments for SQL Functions
Today, we introduce the new availability of named arguments for SQL functions. With this feature, you can invoke functions in more flexible ways...
Introducing Python User-Defined Table Functions (UDTFs)
Apache Spark™ 3.5 and Databricks Runtime 14.0 have brought an exciting feature to the table: Python user-defined table functions (UDTFs). In this blog...
Arrow-optimized Python UDFs in Apache Spark™ 3.5
In Apache Spark™, Python User-Defined Functions (UDFs) are among the most popular features. They empower users to craft custom code tailored to their...
Announcing MLflow 2.8 LLM-as-a-judge metrics and Best Practices for LLM Evaluation of RAG Applications, Part 2
Today we're excited to announce MLflow 2.8 supports our LLM-as-a-judge metrics which can help save time and costs while providing an approximation of...
Big Book of MLOps Updated for Generative AI
Last year, we published the Big Book of MLOps, outlining guiding principles, design considerations, and reference architectures for Machine Learning Operations (MLOps). Since...
Llama 2 Foundation Models Available in Databricks Lakehouse AI
We’re excited to announce that Meta AI’s Llama 2 foundation chat models are available in the Databricks Marketplace for you to fine-tune and...
A Pattern for the Lightweight Deployment of Distributed XGBoost and LightGBM Models
A common challenge data scientists encounter when developing machine learning solutions is training a model on a dataset that is too large to...
Announcing the Public Preview of AI/BI Dashboards!
We are excited to announce the public preview of the next generation of Databricks SQL dashboards, dubbed AI/BI Dashboards . Available today, this...
Deploy Private LLMs using Databricks Model Serving
We are excited to announce public preview of GPU and LLM optimization support for Databricks Model Serving! With this launch, you can deploy...
Making Spark Accessible: My Databricks Summer Internship
My summer internship on the PySpark team was a whirlwind of exciting events. The PySpark team develops the Python APIs of the open...