What to Do When Your Job Goes OOM in the Night (Flowcharts!)
On Demand
Type
- Session
Format
- Hybrid
Track
- Data Engineering
Room
- Moscone South | Level 2 | 211
Duration
- 35 min
Overview
Have you ever had a Spark job just stop working? No idea where to start debugging? Or maybe your job that used to be completed in minutes is now taking hours? Or are you just tired of answering user questions? Come join us for a fun detour into the world of out of memory exceptions, slow jobs, and other things that make our lives sad and leave with techniques to make our lives happy again. This flowchart is based on the initial work of Anya's Spark tuning flowchart updated with our collective experience fixing broken Spark jobs. The talk will wrap up with the methodology we used and how you can contribute to the flowchart (aka guilt you into writing pull requests).
See the best of Data+AI Summit
Watch on demand