spark-ec2 is a handy little tool for spinning up Spark clusters on EC2, but it has a few frustrating problems that are difficult to solve within its current architecture. In this talk, Nick will give an overview of Flintrock, a single-purpose command-line tool for launching and interacting with Spark clusters on EC2. Flintrock is open source and aims to be the spiritual successor to spark-ec2.
Nick has been working with (and occasionally contributing to) Apache Spark since 0.9.0. He currently works on the Data Engineering team at MassMutual and has had stints at Databricks, the Recurse Center, and Turbine/WB Games. Nick is also the author of Flintrock, a command-line tool for launching Apache Spark clusters on EC2.