Session

Virtual Delta Tables: Building a Multimodal Lakehouse

Overview

ExperienceIn Person
TrackArtificial Intelligence & Agents
IndustryEnterprise Technology, Communications, Media & Entertainment, Financial Services
TechnologiesAI/BI, Unity Catalog, Databricks Agents
Skill LevelIntermediate

Managing multimodal inference is one of the defining challenges facing AI teams. LLMs now consider documents, images, and videos, producing higher quality results and reducing time wasted with data prep. The lakehouse architecture solves part of the problem but many teams are struggling to incorporate an increasing variety of data into their workflow. Rather than ingesting, re-encoding, or modifying massive troves of existing data, multimodal teams are looking to work with original artifacts.

 

This talk introduces Virtual Delta Tables and the associated open source code designed for multimodal inference. Virtual Delta Tables incorporate original data from different sources into a singular logical structure for ease of use by data scientists and ML engineers. By offering a Delta Lake interface the entire ecosystem from Databricks to DuckDB can be used in multimodal work. We'll highlight just how far you can push the lakehouse architecture in the rapidly changing AI data landscape!

Session Speakers

Tyler Croy

/Slop Janitor
Scribd, Inc.

Full Summary

Rethinking multimodal data at scale: why Delta Lake can be the API

Modern teams want to ask questions of PDFs, images, and mixed media with the same ease as querying tables. Storage realities, like billions of objects in S3 and strict cost and performance constraints, complicate that goal. The talk argues for treating Delta Lake not just as a storage format but as a simple, universal protocol that can expose everything an organization knows about a document to the tooling people already use on the Databricks Platform. Teams exploring how to unify structured and unstructured knowledge will find that the distinction between structured and unstructured data is precisely what makes this architectural challenge worth solving carefully.

FAQ


It works at small scale, then breaks. Identical-looking documents drift from byte equivalence across tables, Parquet's column skipping loses value when rows contain large blobs, and teams pay for redundant storage as content is copied repeatedly.

It represents external files with offset and size, enabling engines to bin-pack many small files into larger objects while preserving a table-like abstraction. That reduces per-object overhead and API costs, although read amplification across many referenced objects remains a risk.

It is a service that speaks the Delta Lake protocol and S3 API while fabricating the transaction log and Parquet files on demand from another source. Clients see a normal Delta Lake table, but the underlying data can live in a database, object store, or a compute layer that generates artifacts when queried.

Custom actions in the transaction log can be dropped by standard Delta Lake writers running optimize, vacuum, or checkpointing. Approaches that diverge from Parquet data files also violate reader assumptions, so they are viable only if you control the entire stack.

Precomputation is no longer a prerequisite. Analysts query through Delta Lake, the platform generates required artifacts once and caches them, and results become accessible via the same interface for future use. The approach unifies streaming ingestion and backfills around shared generation code.