The big data platform is evolving to be heterogeneous while the dark silicon is coming. As a candidate, FPGA has been noticed across the industry because of its performance-per-power efficiency, re-programmable flexibility and wide range of applicableness. Various IP developed on FPGA could potentially boost growing big data and AI workload on the platform. However, there are gaps to adopt FPGA in big data platform like resource scheduling, isolation and so on. In this session, we would like to introduce a new feature in Yarn to treat FPGA as 1st class citizen resource in Yarn.
We will explain the idea in detail that covers how it works, what the feature provided, and why you should use FPGA-as-a-service in big data platforms such as Yarn or Kubernetes. By using this feature, big data applications can request FPGA resources easier and use them exclusively. Furthermore, we will give some examples such as basic matrix calculation, data compression/decompression, and deep learning workloads to demonstrate how these workloads can get the advantages from this feature. And what we are experimenting to put the same features in Docker/Kubernetes environment.
In this topic, the audience can learn below things:
How FPGA as 1st citizen in Yarn(YARN-5983)/Kubernetes works
Current status and issues update -Our experience and customer case sharing
Session hashtag: #SAISEco9
Weiting is a senior software engineer at Intel Software. He has worked for Big Data and Cloud Solutions including Spark, Hadoop, OpenStack, and Kubernetes for more than 5 years. He has also worked for big data and Intel architecture technologies research including CPU, GPU, and FPGA. One major responsibility for him is to research & optimize Big Data technology and enable global customers to use Big Data with Intel solutions. Weiting is working on next-generation big data technologies on Intel x86 platform.