Session

Sponsored by: definity | From Observability to Agentic Data Engineering: What It Actually Takes

Overview

ExperienceIn Person
TrackData Engineering & Streaming
IndustryEnterprise Technology, Communications, Media & Entertainment, Financial Services
TechnologiesDatabricks SQL, Unity Catalog
Skill LevelIntermediate

Data observability is now standard in lakehouse platforms, but it stops at reactive monitoring. As teams push toward proactive agentic workflows, LLMs alone aren’t enough. Without production-aware context, ability to take action in-flight, and automated guardrails, AI remains limited to suggestive copilots trained on generic data. This session breaks down what it takes to build agentic capabilities to continuously optimize cost, resolve incidents in-motion, and evolve pipeline code. We’ll show how definity enables them through full-stack context across data, pipelines, and infrastructure; runtime execution control; and built-in validation loops – turning signals into autonomous action inside Spark pipelines.

Session Speakers

Roy Daniel

/Co-Founder & CEO
definity