Sponsored by: Prophecy | Designing a Copilot for Data Transformation with Spark and SQL
OVERVIEW
EXPERIENCE | In Person |
---|---|
TYPE | Breakout |
TRACK | Data Engineering and Streaming |
INDUSTRY | Professional Services |
TECHNOLOGIES | Apache Spark, ETL, GenAI/LLMs |
SKILL LEVEL | Beginner |
DURATION | 40 min |
AI holds the promise of simplifying and accelerating nearly every aspect of data management. Github has a copilot for general purpose programming. But what about data users? What is the right copilot for them? Sprinkling AI capabilities on top of existing products or converting natural language to code is not sufficient - it’s incremental thinking. It is time to ask ourselves: How would one design a copilot for data transformation, from scratch? Join us as we discuss what makes data transformation unique, and how we’ve designed a copilot that mixes the right ingredients into a copilot that will make every data user 10x more productive. We’ll also show various LLMs applied to programming - the results in accuracy and latency, and what that means for the copilot for the data user. This session will include demos you won’t want to miss!
SESSION SPEAKERS
IMAGE COMING SOON
Raj Bains
/Founder and CEO
Prophecy
Maciej Szpakowski
/Co-Founder & CTO
Prophecy