HomepageData + AI Summit 2022 Logo
Watch on demand

Monitoring and Quality Assurance of Complex ML Deployments via Assertions

On Demand

Type

  • Session

Format

  • Hybrid

Track

  • Data Science, Machine Learning and MLOps

Difficulty

  • Advanced

Room

  • Moscone South | Upper Mezzanine | 156

Duration

  • 35 min
Download session slides

Überblick

Machine Learning (ML) is increasingly being deployed in complex situations by teams. While much research effort has focused on the training and validation stages, other parts have been neglected by the research community.



In this talk, Daniel Kang will describe two abstractions (model assertions and learned observation assertions) that allow users to input domain knowledge to find errors at deployment time and in labeling pipelines. He will show real-world errors in labels and ML models deployed in autonomous vehicles, visual analytics, and ECG classification that these abstractions can find. I'll further describe how they can be used to improve model quality by up to 2x at a fixed labeling budget. This work is being conducted jointly with researchers from Stanford University and Toyota Research Institute.

Session Speakers

Daniel Kang

PhD student

Stanford University

Das Beste des Data+AI Summits anzeigen

Watch on demand