Packaging Spark for Fedora

Download Slides

High-quality downstream distributions of open-source projects benefit everyone. End-users enjoy convenient installation and upgrades, dependency management, system integration, and the fruits of a thriving testing and support community. Downstream packagers contribute testing and fixes to upstream developers and free up core teams to focus on enhancements and fixes rather than on the details of packaging. In this talk, we’ll discuss these benefits and present our efforts — along with the Fedora Big Data SIG — to package Spark for Fedora. We’ll cover some of the unique challenges presented by the impedance mismatch between traditional downstream packaging models and the Scala and big data ecosystems, present our current progress, and discuss opportunities for other members of the community to get involved.

« back
About William Benton

William Benton is passionate about making it easier for machine learning practitioners to benefit from advanced infrastructure and making it possible for organizations to manage machine learning systems. His recent roles have included defining product strategy and professional services offerings related to data science and machine learning, leading teams of data scientists and engineers, and contributing to many open source communities related to data, ML, and distributed systems. Will was an early advocate of building machine learning systems on Kubernetes and developed and popularized the “intelligent applications” idiom for machine learning systems in the cloud. He has also conducted research and development related to static program analysis, language runtimes, cluster configuration management, and music technology.