The MLOps Platform at WGU: Solutions to Production ML with Databricks

WGU's MLOps platform, MARVIN, tackles challenges in deploying ML models with Databricks at scale to products that serve a student population of almost 170,000. Despite ML benefits, issues like source control, monitoring, and standardized procedures hindered progress. MARVIN, comprising a Python package, a project template repo, and Databricks integration, addresses these challenges. The python package streamlines data scientists' experience which includes wrappers for MLflow and monitoring tools. The project template ensures consistency by generating new project repositories with base code, CICD configurations, and permissions management. This presentation provides insights into MARVIN's development, emphasizing its transformative role and offering strategies for data science workflows. Valuable for professionals in data science, engineering, and machine learning.