SESSION

Goodbye to Cluster Wait Time: Hello Lightning-fast R/Py Library Loads!

Accept Cookies to Play Video

OVERVIEW

EXPERIENCEIn Person
TYPELightning Talk
TRACKData Science and Machine Learning
INDUSTRYEnterprise Technology, Health and Life Sciences, Media and Entertainment
TECHNOLOGIESDatabricks Experience (DBX), AI/Machine Learning, Orchestration
SKILL LEVELIntermediate
DURATION20 min
DOWNLOAD SESSION SLIDES
Ever wished you could start up an interactive cluster without having to wait for libraries to be pulled down from repositories and installed before you could begin your work? You're not alone! Many people find themselves in this same situation, and to ‘save’ time they multitask while waiting, only to have the cluster terminate after a certain amount of time of inactivity, forcing them to start over. Imagine being able to skip this waiting game and have all the required libraries available right from the start. This can be a reality for R packages as well as Python libraries. The approach can also be applied to other DSML developments and workflows. There is a productivity hack that can have you load up libraries in under 60 seconds whenever your cluster spins up!

SESSION SPEAKERS

May Merkle-Tan

/Sr. Specialist Solutions Engineer
Databricks