Session

AI Will Go Wrong and the Blueprint to Get It Right

Overview

Experience	In Person
Track	Data Strategy
Industry	Healthcare & Life Sciences, Manufacturing, Financial Services
Technologies	Unity Catalog, Databricks Agents, Lakebase
Skill Level	Intermediate

Most AI failures aren't technology problems — they're people, process, and architecture problems. In this hands-on session, you'll witness a live AI trainwreck: an agent operating without guardrails, governance, or the right organizational design. Then, in a real-time post-mortem, we will collectively dissect what broke and why. You'll map the missing middle layer: the intersection of roles, accountability structures, and platform architecture. And you'll work alongside peers to design practical fixes. The session closes with an exercise where attendees act as domain experts to train an AI Judge capable of scaling oversight across enterprise deployments. Expect minimal slides, maximum doing, and a concrete blueprint you can apply within 30 days. This session is built for executives who are tired of theoretical AI frameworks and want proven patterns that make AI projects safer, faster, and genuinely business-ready.

Session Speakers

Lexy Kassan

/Lead Data & AI Strategist
Databricks

Maria Zervou

/Chief AI Officer - EMEA
Databricks

Full Summary

Why AI Fails in the Boardroom: Building Trust Through People, Process, and Judges

Many enterprise AI projects reach production yet stall in adoption because stakeholders do not trust the outputs. The session argues that trust, not model accuracy alone, determines whether AI drives real business value.

The core remedy is to encode business judgment around AI through people, process, and measurable evaluations, a playbook that aligns with the approach in how leading companies are delivering trusted AI-powered self-service analytics.

A vivid acquisition case shows how an off-the-shelf AI can look confident and quantitative, yet collapse under scrutiny when assumptions are stale, double counted, or ownerless. Confidence rises only when domain expertise, governance, and independent checks surround the model and make decision logic auditable.

Trustworthy AI is a business outcome, not a technical milestone

Success means adoption, impact, and scale, not just getting a model into production. Stakeholders need to believe outputs represent organizational ground truth and current context, especially in high-stakes settings.

Roughly 70 percent of trust comes from people and process. Executive sponsorship sets ambition and risk appetite, risk management defines assessment and mitigation, and change management adjusts ways of working so teams use AI daily. Without leadership that protects SME time and sets clear approval gates, initiatives stall.

Rebalancing the stack: foundations and guardrails before models

Organizations overinvest in model choice and underinvest in the scaffolding that makes outputs usable. The durable order of operations is clear: start with curated corporate data, lineage, security, and auditability, then add architecture-level guardrails and observability, and only then select models. The model is the smallest piece.

Domain experts, business and data analysts, and model stewards supply the semantic layer, business definitions, and ground truth. On the Databricks Platform, those definitions become metric views and ontologies that align AI behavior with how the business measures itself.

Why the first AI answer failed a board test

A packaged executive AI recommended acquiring a software firm for 340 million dollars based on financial strength, customer health, market opportunity, and ROI. On inspection, market sizing leaned on a single stale source, revenue synergies were double counted, and no accountable owners had validated methods. Once sales, strategy, finance, and corporate development reviewed the logic, the defensible price dropped to 208 million dollars and confidence moved to under review.

The model was not inherently wrong. It lacked a mechanism to encode what the business considers true, current, and acceptable risk.

Codifying judgment as evaluation judges

Trust emerges when organizations translate business knowledge into explicit evaluation judges that wrap AI inputs and outputs. Pre-processed judges verify scope before the AI runs, such as data freshness, source independence, definitional consistency, and method ownership. Post-processed judges validate outputs against thresholds, assumptions, and evidence requirements, then route results to a human review gate.

Applied to the acquisition brief, judges passed financial strength, flagged customer concentration, failed market opportunity due to stale, single-sourced research, and appropriately lowered overall confidence. No retraining or pipeline changes were required. Judges formed a governance layer that delivered certified briefs with provenance, audit trails, and sign-offs.

From confident prototypes to credible decisions

Enterprises that formalize a durable loop between domain experts, evaluation criteria, and system behavior can move AI from demo to boardroom. Invest first in people and process, then encode that judgment as pre and post checks around the model. Adoption follows trust, and trust follows clear ownership, thresholds, and evidence.

FAQ

Ideally yes, because early governance accelerates trust and adoption. If a system is already live, add judges incrementally and learn from real outputs to refine criteria over time.

Pre-processed judges validate inputs and methods before the AI runs, checking source freshness, independence, and definitional consistency. Post-processed judges validate outputs against business thresholds and acceptance criteria, flagging exceptions before a human reviewer sees them.

Frame involvement as ongoing stewardship, not a one-time knowledge dump. Pair a business-side champion with visible executive sponsorship so SME time is protected and their expertise shapes the system that will support their team.

Teams skip defining success upfront, both for adoption and acceptable accuracy or risk thresholds. Another frequent miss is tracking purely technical metrics, such as latency, without tying them to business outcomes like retention, revenue, or decision quality.

No. Judges wrap business logic around existing inputs and outputs, so teams can retrofit governance onto purchased tools or custom agents without modifying models or pipelines. The session's example applied judges as a layer on top of a SaaS app.