Kent Buenaventura, is a senior member of Unity’s Analytics Team, helping build and re-architect the big data infrastructure. Previously, he worked on building scalable game servers at DeNA and developed games at Sunstorm Interactive.
October 25, 2016 05:00 PM PT
Unity is one of the largest game development platforms. We analyze billions of events daily. Our legacy data infrastructure is based on Hive and MapReduce. Having reached a scalability barrier we decided to re-architect our infrastructure to use Spark as the primary processing engine. It supports our realtime and on-demand processing needs. In this talk we will present how we approached migrating from Hive to Spark and the issues that we faced along the way.