This blog post is the second in a series on ML platforms, operations, and governance. For the first post, see Rafi Kurlansik’s post...

Try the Hyperopt notebook to reproduce the steps outlined below and watch our on-demand webinar to learn more. Hyperopt is one of the...

Try this notebook in Databricks On June 20th, our team hosted a live webinar— Automated Hyperparameter Tuning, Scaling and Tracking on Databricks —with...

Hyperparameter tuning is a common technique to optimize machine learning models based on hyperparameters, or configurations that are not learned during model training...

We are excited to announce the release of Databricks Runtime 5.4 ML ( Azure | AWS ). This release includes two Public Preview...

We are excited to announce the release of Databricks Runtime 5.2 for Machine Learning. This release includes several new features and performance improvements...

Today, we are excited to introduce HorovodRunner in our Databricks Runtime 5.0 ML ! HorovodRunner provides a simple way to scale up your...

Thanks to our awesome interns! This summer, our Engineering interns at Databricks did amazing work. Our interns, working on teams from Developer Tools...

Developing custom Machine Learning (ML) algorithms in PySpark—the Python API for Apache Spark—can be challenging and laborious. In this blog post, we describe...

This is a cross blog post effort between Databricks and Uber Engineering. Yun Ni is a software engineer on Uber’s Machine Learning Platform...

Try this notebook on Databricks Intel recently released its BigDL project for distributed deep learning on Apache Spark. BigDL has native Spark integration...

We are excited to announce the general availability of Graphic Processing Unit (GPU) and deep learning support on Databricks! This blog post will...

Last week, we held a live webinar, Apache Spark MLlib 2.x: Migrating ML Workloads to DataFrames , to demonstrate the ease with which...

Databricks is adding support for Apache Spark clusters with Graphics Processing Units (GPUs), ready to accelerate Deep Learning workloads. With Spark deployments tuned...

Introduction Consider these Machine Learning (ML) use cases: A data scientist produces an ML model and hands it over to an engineering team...

Introduction Apache Spark is fast, but applications such as preliminary data exploration need to be even faster and are willing to sacrifice some...

Introduction Graph structures are a more intuitive approach to many classes of data problems. Whether traversing social networks, restaurant recommendations, or flight paths...

We would like to thank Ankur Dave from UC Berkeley AMPLab for his contribution to this blog post. Databricks is excited to announce...

Data scientists often spend hours or days tuning models to get the highest accuracy. This tuning typically involves running a large number of...

To learn more about Apache Spark, attend Spark Summit East in New York in Feb 2016 . With the latest release, Apache Spark’s...

To try the new visualization features mentioned in this blog, sign up for a 14-day free trial of Databricks today. You've built your...

This blog was written by Feynman Liang and Joseph Bradley from Databricks, and Yuhao Yang from Intel. To get started using LDA, download...

Apache Spark 1.2 introduced Machine Learning (ML) Pipelines to facilitate the creation, tuning, and inspection of practical ML workflows. Spark’s latest release, Spark...

Topic models automatically infer the topics discussed in a collection of documents. These topics can be used to summarize and organize documents, or...

This is a post written together with Manish Amde from Origami Logic. Apache Spark 1.2 introduces Random Forests and Gradient-Boosted Trees (GBTs) into...

MLlib’s goal is to make practical machine learning (ML) scalable and easy. Besides new algorithms and performance improvements that we have seen in...

This is a post written together with one of our friends at Origami Logic. Origami Logic provides a Marketing Intelligence Platform that uses...