HomepageData + AI Summit 2023 Logo
JUNE 26-29, 2023
SAN FRANCISCO + VIRTUAL
Attend Live

What to Do When Your Job Goes OOM in the Night (Flowcharts!)

On Demand

Type

  • Session

Format

  • Hybrid

Track

  • Data Engineering

Room

  • Moscone South | Level 2 | 211

Duration

  • 35 min
Download session slides

Overview

Have you ever had a Spark job just stop working? No idea where to start debugging? Or maybe your job that used to be completed in minutes is now taking hours? Or are you just tired of answering user questions? Come join us for a fun detour into the world of out of memory exceptions, slow jobs, and other things that make our lives sad and leave with techniques to make our lives happy again. This flowchart is based on the initial work of Anya's Spark tuning flowchart updated with our collective experience fixing broken Spark jobs. The talk will wrap up with the methodology we used and how you can contribute to the flowchart (aka guilt you into writing pull requests).

Session Speakers

Headshot of Holden Karau

Holden Karau

Engineer

Netflix

Headshot of Anya Bida

Anya Bida

Technical Evangelist

prophecy.io

See the best of Data+AI Summit

Watch on demand