Classic to Serverless: An Agentic Migration Playbook for Notebooks, Jobs, and Pipelines
Overview
| Experience | In Person |
|---|---|
| Track | Data Engineering & Streaming |
| Industry | Enterprise Technology |
| Technologies | Lakeflow |
| Skill Level | Intermediate |
Moving real workloads from classic to serverless compute can mean dozens of small changes: Spark configs that no longer apply, init scripts that need rethinking, Scala and JAR jobs, and pipelines tied to specific runtime versions. Doing this by hand across hundreds of Jobs and Notebooks does not scale.
This session is a hands-on practitioner's guide to migrating Notebooks, Jobs, and Pipelines to serverless, with a focus on the new agentic tooling that automates most of the work. We walk through concrete examples covering Python and Scala/JAR workloads, show how the migration agent inspects code, classifies incompatibilities, and proposes fixes, and demonstrate how to validate the result before promoting the change.
Attendees leave with a repeatable migration playbook, a clear view of what is fully automated today versus where a human stays in the loop, and a sense of which workloads are ready to move now.
Session Speakers
Achille Negrier
/Associated Product Manager
Databricks
Prashanth Babu Velanati Venkata
/Product Specialist
Databricks