Apache Spark has revolutionized big data processing and analytics with its great ecosystem, strong community and compelling collection of extensions and packages. It offers API in Scala, Java, Python and R for application development covering a large population of developers. It was, however, out of reach for C# developers. Mobius project introduces C# as another first-class Spark programming language that support the same Spark RDD, DataFrame and Streaming APIs. Organizations that use C# as the primary application development language can start building Spark applications, and reuse existing .NET libraries in new Spark applications. We have received great support from C# lovers and positive feedback from the Spark community. In this presentation, we will walk through writing Spark driver program in C#, explain internal architecture of Mobius, share the observations of C# applications running in Spark cluster, and recommend best practices. Mobius is open-sourced @ https://github.com/microsoft/Mobius.
Learn more: