Session

Building End-to-End ML Systems With Genie Code

Overview

Experience	In Person
Track	Artificial Intelligence & Agents
Industry	Enterprise Technology, Retail & Consumer Goods, Financial Services
Technologies	Genie
Skill Level	Intermediate

Coding agents are transforming how data science teams build and operate machine learning systems. In this session, we'll explore how teams use Genie Code across the ML lifecycle—across EDA, feature engineering, model development, evaluation, deployment, and monitoring.

We'll also provide an inside look at how Genie Code was built and optimized for ML workflows. Genie Code understands your data through Unity Catalog and leverages metadata, business semantics, governance policies, and lineage to provide context-aware assistance. Deep integration across the Databricks ML stack enables teams to move faster and work with greater confidence.

We'll then look ahead at how Genie Code is evolving from a coding assistant into an agent that can help deploy, serve, and operate ML systems in production. You'll also learn how to customize Genie Code for your ML team through skills and instructions, as well as other tips and tricks to get the most out of Genie Code.

Attendees will leave with practical examples of how coding agents are reshaping machine learning and actionable guidance for applying these capabilities to their own ML projects.

Session Speakers

IMAGE COMING SOON

Nemanja Petrović

/Principal Software Engineer
Databricks

IMAGE COMING SOON

Tejas Sundaresan

/Staff Product Manager
Databricks

Full Summary

Genie Code for ML: How agents are reshaping the machine learning lifecycle on Databricks

Most ML teams do not lack ideas, they lack time to operationalize them. The session introduces Genie Code for ML, an agentic assistant integrated across the Databricks Platform that accelerates work from exploration to deployment and operations. The core claim is simple: when an agent understands your data, infrastructure, and team workflows, it can eliminate much of the hidden work that stalls models before production.

As explored in Databricks' perspective on AI-powered analytics and data intelligence, the training code is only a small slice of real-world ML delivery. Teams serious about closing that gap can find a practical foundation in Databricks' overview of machine learning on the platform.

Context is the unlock: data, infrastructure, and workflow awareness

An agent that only reads code cannot fix ML bottlenecks. Three forms of context matter most. Data context, including quality, freshness, governance, class imbalance, and leakage. Infrastructure context, including the right compute type, GPU utilization, serving reliability, and cost. Workflow context, including retraining triggers, drift responses, and evaluation standards.

Genie Code for ML is designed around these layers. It embeds Databricks ML best practices, integrates with Unity Catalog, MLflow, the feature store, Lakeflow, and model serving, and is guided by Genie Ontology for a shared understanding of business metrics, governance rules, and prior experiments. The result is an agent that reasons about your environment, not generic snippets.

One continuous path from authoring to serving

The demo centers on a credit card fraud detection workflow. Starting with a governed table reference, Genie Code samples data, runs exploratory analysis, engineers features, and trains a classifier that accounts for heavy class imbalance. It produces correlation checks to guard against label leakage and logs the full run to MLflow.

Continuity is the differentiator. Inside MLflow, the agent reviews experiment runs, recommends additional metrics tailored to skewed data, such as dollar-weighted fraud detection rate, and edits the source Databricks Notebook in place to add them. At serving, it inspects a live endpoint, correlates provisioned capacity with traffic, and flags over-provisioning. One assistant carries context across authoring, experimentation, and serving, which makes end-to-end acceleration credible.

Evaluation that scales with agent-authored code

As agents write more ML code, manual review cannot keep pace. The team adopts LLM-as-judge with plain-English rubrics shared with human raters, then refines machine rubrics using prompt evolution methods like DSPY and GEPA. Because Databricks Notebook evaluation is costly and these techniques often need many training samples, Databricks introduced Memaligned, an open-source MLflow library that aligns LLM judges to human scores using very few samples. Reported results show reductions in mean absolute error between human and machine evaluations, enabling CI and real-time alerting to catch regressions in generated code and models. When divergence reappears, teams can run quick re-alignment rounds.

Customization and ZeroOps extend impact

Genie Code adapts to team standards and tools. Integrations with MCP servers such as Slack, Google Docs, and Jira let teams route alerts, pull business semantics, and track issues. Instruction files and user-authored skills (markup files loaded on demand) encode preferred libraries, code styles, and domain logic. Awareness of catalogs, schemas, clusters, runtimes, and Python environments powers better defaults and targeted hints.

Genie ZeroOps for ML brings the same agentic approach to operations. It continuously watches production models, raises alerts on model and infrastructure metrics, investigates incidents, and helps diagnose and repair issues. It also serves as a central hub for operational health across a growing model portfolio, with availability targeted later in the summer following the session.

Turning ML backlogs into shipped systems

Agents only become useful when they understand data, infrastructure, and workflow together. Genie Code for ML is available now, and Genie ZeroOps for ML is coming soon. Together they aim to convert ambitious ML backlogs into production systems by carrying context across the entire lifecycle.

FAQ

It integrates with Databricks Notebooks, Unity Catalog, MLflow experiments and models, the declarative feature store, model serving endpoints on CPU or GPU, and Lakeflow Declarative Pipelines. One agent can move from data exploration through training, evaluation, and deployment without losing context between tools.

The agent is aware of your workspace, attached clusters, runtime, and Python environment. It can suggest serverless CPU, serverless GPU, or custom clusters based on the workload and data, and it can flag issues such as over-provisioned serving endpoints relative to real traffic.

Agent-authored models and notebooks grow faster than humans can review. Misaligned LLM judges let regressions slip through. Memaligned, an open-source library in MLflow, aligns machine judges to human rubrics with few samples, lowering error and enabling CI-style regression checks and real-time alerts.

Yes. Teams can add MCP server integrations for Slack, Google Docs, and Jira, upload instruction files that encode preferred practices, and author custom skills as markup files that Genie Code loads on demand. The agent then applies these preferences when generating and editing code.

Genie ZeroOps for ML applies agents to operations. It monitors production models, investigates alerts on model and infrastructure metrics, and assists with diagnosis and repair. The speakers indicated it is planned for release later in the summer after the session.