SESSION

Semi-Structured Communication Data with Delta Lake and Microservices

Accept Cookies to Play Video

OVERVIEW

EXPERIENCEIn Person
TYPEBreakout
TRACKData Engineering and Streaming
INDUSTRYEnterprise Technology, Media and Entertainment
TECHNOLOGIESDelta Lake
SKILL LEVELIntermediate
DURATION40 min
DOWNLOAD SESSION SLIDES

Our talk presents a solution for efficiently storing and managing semi-structured communication data for eDiscovery. With the rise of short-form messaging platforms, the volume and complexity of data used in legal processes have significantly increased. Delta Lake is particularly suited to handle the complexities and requirements of eDiscovery data. Our approach involves developing microservices that provide abstracted layers of data access, enabling efficient querying, processing, and analysis of semi-structured communication data using technologies like delta-rs, Arrow Flight, and DataFusion. Our solution is a strategic response to the changing landscape of digital communication and its implications for legal processes. Our talk will provide details on the implementation and integration of Delta Lake and these microservices, of interest to software engineers, data architects, and anyone interested in technology and the legal industry.

SESSION SPEAKERS

Alex Wilcoxson

/Staff Software Engineer
Relativity

Greg Ott

/Staff Software Engineer
Relativity