HomepageData + AI Summit 2022 Logo
Watch on demand

DataFusion and Arrow: Supercharge Your Data Analytical Tool with a Rusty Query Engine

On Demand

Type

  • Session

Format

  • Hybrid

Track

  • Data Engineering

Difficulty

  • Intermediate

Room

  • Moscone South | Upper Mezzanine | 155

Duration

  • 35 min
Download session slides

Overview

Rust, the Apache Arrow project, and the Data Fusion Query Engine are increasingly being used to accelerate the creation of modern data stacks due to their flexible design, blazing performance, and permissive licensing.

This talk will:
* Briefly review the benefits of Rust and Apache Arrow
* Present an overview of the Data Fusion Query Engine
* Showcase several systems built using DataFusion, and why they chose to do so. Some, such as ROAPI use the full DataFusion stack; Some such as InfluxDB IOx extensively leverage plan extension points to customize the behavior, and other projects only use the physical execution layer.

Session Speakers

Andrew Lamb

Staff Engineer

InfluxData

Daniël Heres

DATA / MACHINE LEARNING ENGINEER

GoDataDriven

See the best of Data+AI Summit

Watch on demand