Platform blog

Recap of Databricks Machine Learning announcements from Data & AI Summit

Share this post

Databricks Machine Learning on the lakehouse provides end-to-end machine learning capabilities from data ingestion and training to deployment and monitoring, all in one unified experience, creating a consistent view across the ML lifecycle and enabling stronger team collaboration. At the Data and AI Summit this week, we announced capabilities that further accelerate ML lifecycle and production ML with Databricks. Here’s a quick recap of the major announcements.

MLflow 2.0 Including MLflow Pipelines
MLflow 2.0 is coming soon and will include a new component, Pipelines. MLflow Pipelines provides a structured framework that enables teams to automate the handoff from exploration to production so that ML engineers no longer have to juggle manual code rewrites and refactoring. MLflow Pipeline templates scaffold pre-defined graphs with user-customizable steps and natively integrate with the rest of MLflow’s model lifecycle management tools. Pipelines also provide helper functions, or “step cards”, to standardize model evaluation and data profiling across projects. You can try out a Beta version of MLflow Pipelines today with MLflow 1.27.0.

MLflow 2.0 introduces MLflow Pipelines - pre-defined, production ready templates to accelerate ML
MLflow 2.0 introduces MLflow Pipelines - pre-defined, production-ready templates to accelerate ML

Serverless Model Endpoints
Deploy your models on Serverless Model Endpoints for real-time inference for your production applications. Serverless Model Endpoints provide highly available, low latency REST endpoints that can be set up and configured via UI or API. Users can customize autoscaling to handle their model’s throughput and for predictable traffic use cases, and teams can save costs by autoscaling all the way down to 0. Serverless Model Endpoints also have built-in observability so you can stay on top of your model serving. Now your data science teams don’t have to build and maintain their own kubernetes infrastructure to serve ML models. Sign up now to get notified for the Gated Public Preview.

Serverless Model Endpoints provide production-grade model serving hosted by Databricks
Serverless Model Endpoints provide production-grade model serving hosted by Databricks

Model Monitoring
Track the performance of your production models with Model Monitoring. Our model monitoring solution auto-generates dashboards to help teams view and analyze data and model quality drift. We also provide the underlying analysis and drift tables as Delta tables so teams can join performance metrics with business value metrics to calculate business impact as well as create alerts when metrics have fallen below specified thresholds. While Model Monitoring automatically calculates drift and quality metrics, it also provides an easy mechanism for users to incorporate additional custom metrics. Stay tuned for the Public Preview launch…

Monitor your deployed models and the related data that feeds them all in a centralized location with auto-generated dashboards and alerting.
Monitor your deployed models and the related data that feeds them all in a centralized location with auto-generated dashboards and alerting.

Learn more:

Try Databricks for free

Related posts

Company blog

A Guide to Data + AI Summit Sessions: Machine Learning, Data Engineering, Apache Spark and More

April 26, 2021 by Jules Damji in Company Blog
We are only a few weeks away from Data + AI Summit, returning May 24–28. If you haven’t signed up yet, take advantage of...
Platform blog

Top 5 Workflows Announcements at Data + AI Summit

The Data and AI Summit was chock-full of announcements for the Databricks Lakehouse platform. Among these announcements were several exciting enhancements to Databricks...
See all Platform Blog posts