Petar started out as a Java developer almost 20 years ago, and worked as a Software Architect, Team Leader and IBM software consultant. After switching to the exciting new field of Big Data technologies, he wrote the Spark in Action book (Manning 2016) and these days primarily works on Apache Spark and Big Data projects. Today he is CTO of SV Group in Zagreb, Croatia, while also pursuing his PhD at the University of Zagreb. He is collaborating with Astronomy Department at the University of Washington on building new methods for processing images and data from future astronomical surveys.
October 15, 2019 05:00 PM PT
The next decade promises to be exciting for both astronomy and computer science with a number of large-scale astronomical surveys in preparation. One of the most important ones is Large Scale Survey Telescope, or LSST. LSST will produce the first 'video' of the deep sky in history by continually scanning the visible sky and taking one 3.2 giga-pixel image every 20 seconds. In this talk we will describe LSST's unique design and how its image processing pipeline produces catalogs of astronomical objects. To process and quickly cross-match catalog data we built AXS (Astronomy Extensions for Spark), a system based on Apache Spark. We will explain its design and what is behind its great cross-matching performance.