Sayan is a senior applied scientist in the Zillow A.I. team. Sayan’s role is positioned in the intersection of ML and software engineering where he is building the centralized ML system to automate the data quality monitoring and diagnosis process within Zillow. He has several years of experience of productionalizing state of the art Machine Learning and Statistical methods and also proposed many novel algorithms.Sayan received his Ph.D. in Statistics from Michigan State University in 2016 and worked at Dell and Tibco Software before joining Zillow.
May 27, 2021 05:00 PM PT
Organizations rely heavily on time series metrics to measure and model key aspects of operational and business performance. The ability to reliably detect issues with these metrics is imperative to identifying early indicators of major problems before they become pervasive. This is a difficult machine learning and systems problem because temporal patterns are complex, ever changing, and often very noisy, traditionally requiring significant manual configuration and model maintenance.
At Zillow, we have built an orchestration framework around Luminaire, our open-source python library for hands-off time-series Anomaly Detection. Luminaire provides a suite of models and built-in AutoML capabilities which we process with Spark for distributed training and scoring of thousands of metrics. In this talk, we will cover the architecture of this framework and performance of the Luminaire package across detection and prediction accuracy as well as runtime efficiency.