SESSION
Delta Merge Optimizations with Jodie Helpers
OVERVIEW
| EXPERIENCE | In Person |
|---|---|
| TYPE | Lightning Talk |
| TRACK | Data Lakehouse Architecture |
| INDUSTRY | Enterprise Technology, Professional Services |
| TECHNOLOGIES | Delta Lake |
| SKILL LEVEL | Intermediate |
| DURATION | 20 min |
| DOWNLOAD SESSION SLIDES | |
The talk will primarily revolve around Delta Merge Optimizations and the contributions we made to the Jodie repo:
- Delta Merge Optimization Strategies: https://medium.com/@joydeep.roy/delta-merge-optimisation-strategies-b78f18066966
- Change Data Feed implications on Delta tables: Performance Considerations and Failure Scenarios to look out for. Some content would be taken from https://medium.com/@joydeep.roy/delta-merge-optimisation-strategies-b78f18066966, but other strategies would also be covered
- Delta Merge Data Skipping: Based on the contribution made in Jodie - https://github.com/MrPowers/jodie?tab=readme-ov-file#number-of-shuffle-files-in-merge--other-filter-condition
SESSION SPEAKERS
Joydeep Banik Roy
/Head of Data Science and ML Engineering
Zeotap