DataFusion and Arrow: Supercharge Your Data Analytical Tool with a Rusty Query Engine
- Data Engineering
- Moscone South | Upper Mezzanine | 155
- 35 min
Rust, the Apache Arrow project, and the Data Fusion Query Engine are increasingly being used to accelerate the creation of modern data stacks due to their flexible design, blazing performance, and permissive licensing.
This talk will:
* Briefly review the benefits of Rust and Apache Arrow
* Present an overview of the Data Fusion Query Engine
* Showcase several systems built using DataFusion, and why they chose to do so. Some, such as ROAPI use the full DataFusion stack; Some such as InfluxDB IOx extensively leverage plan extension points to customize the behavior, and other projects only use the physical execution layer.