MLflow Pipelines

Automating and Scaling MLOps With MLflow Pipelines

Despite being an emerging technology, MLOps is hard and there are no widely established approaches for managing it.

What makes it even harder is that in many companies, the ownership of MLOps usually falls through the cracks between data science teams and production engineering teams.

Data scientists are mostly focused on modeling business problems and reasoning about data, features and metrics, while production engineers and ops are mostly focused on traditional DevOps for software development, ignoring ML-specific ops like ML development cycles, experiment tracking, and data and model validation.

In this talk, we will introduce MLflow Pipelines, an opinionated approach to MLOps. It provides predefined ML pipeline templates for common ML problems and opinionated development workflows to help data scientists bootstrap ML projects, accelerate model development, and ship production-grade code with little help from production engineers.