Databricks Document Intelligence: pushing the frontier for complex document extraction

Every enterprise has valuable data trapped in messy, unstructured documents. Today, Databricks Document Intelligence helps thousands of customers put their data to work, turning billions of pages into structured data that powers production pipelines, agents, and applications. Customers like Panasonic, EY-Parthenon, and Intercontinental Exchange (NYSE) use Document Intelligence on their most demanding workflows, processing millions of documents weekly.

When working with customers, we noticed several difficult extraction problems where existing large language model (LLM) or rules-based document extraction solutions fall short:

• Long documents. A lease whose page-1 renewal terms depend on a clause on page 80, or an agreement whose page-150 paragraph redefines a term from page 3. Existing solutions fail to resolve these cross-references.
• Large, nested outputs. A multi-page bill of lading with hundreds of SKUs, or an invoice with thousands of line items. Existing solutions drop or truncate fields as outputs grow.
• Complex schemas and reasoning. A risk classification that synthesizes three financial statements, or a contract value that applies listed discounts across every recorded price. Existing solutions fail to consistently apply the correct logic across documents.

Today, we’re excited to introduce Precision Mode in our document extraction API, ai_extract, setting a new bar for accuracy on the most complex enterprise documents and tasks.

Precision Mode combines our custom-trained models for document extraction with an agentic harness to deliver reliable and accurate extraction on long documents, large outputs, and reasoning-heavy schemas. Across benchmarks spanning roughly 9,000 complex documents, Precision Mode achieves the state of the art quality, outperforming the latest frontier models on extract accuracy by a large margin.

"At Intercontinental Exchange, we process millions of complex, highly variable financial documents every month. Document Intelligence helps us turn that complexity into structured market intelligence, enabling us to move faster, deliver greater value to our clients, and unlock agentic workflows that accelerate analysis and decision-making at scale."—Anand Pradhan, CTO and Head of AI, Mortgage Data at Intercontinental Exchange (NYSE)

A New Approach to Complex Document Extraction

To push accuracy on the hardest extraction tasks, our research and engineering teams approached document extraction quality from two layers: customizing the model itself and building an effective agent harness around the model.

• We trained custom, efficient models for document extraction. Working from benchmarks built around difficult customer workloads, our research team trained custom models to find, reason over, and extract structured information from complex documents. Rather than relying on increasingly large general-purpose models, we optimized for the task we want to solve: accurate structured extraction.
• We built an extraction harness designed to overcome model failure modes. Even a strong model can struggle when it has to reason across hundreds of pages or generate thousands of fields at once. Our team built an agent harness, inspired by Databricks MemEx, that semantically decomposes large extraction jobs, executes smaller tasks in parallel, preserves intermediate results, and reconciles them into one final structured output.

Evaluation Methodology

Benchmark Design and Dataset Composition

To validate Precision Mode, we designed our evaluation benchmarks around workloads that push existing approaches to their limits.

Concretely, we evaluated Precision Mode on roughly 9,000 documents spanning the three extraction challenges it was designed to solve. The evaluation includes documents up to 2,000 pages, invoices with thousands of line items, dense multi-page tables and charts, schemas with more than 300 deeply nested fields, and reasoning-heavy tasks that require cross-referencing information across a document.

The documents come from two sets of benchmarks:

• 10 internal datasets inspired by the most difficult customer workloads we’ve seen, spanning key industries including financial services, manufacturing, and healthcare.
• 5 public benchmarks: VAREX, RealDocBench, LongExtractBench, and LEDGER, plus a long-document stress test using the Caselaw Access Project dataset.

Together, these datasets cover documents including 10-K filings, bills of lading, technical manuals, financial documents, clinical notes, government patent and funding applications, and more.

Baseline Design and Model Comparisons

A natural starting point for document extraction is a single frontier-model call: pass in the document and schema, and ask the model to return the structured output. But on the dense and complex workloads we evaluate, that approach quickly breaks down. Long documents can exceed model context limits, causing inaccurate and incomplete results.

So we benchmarked against a stronger, more realistic baseline: chunk-and-merge. We split each document into smaller chunks, extract from each independently, and merge the results into a final output—the same pattern we see engineers use when a single model call isn't enough.

We tested the chunk-and-merge approach using leading GPT, Claude, and Gemini models with their default API settings. Then, we compared each against Precision Mode on extraction accuracy. ¹

Benchmark Results

Across our benchmarks, Precision Mode reaches 94.7% accuracy, outperforming the strongest frontier model chunk-and-merge baseline, GPT-5.6 Sol, by seven points.

Particularly, on difficult long-document workloads, we observed frontier-models encounter numerous operational failure modes including chunk timeouts, truncated outputs, and incomplete final merges that did not conform to the requested schema. On the other hand, Precision Mode’s agentic approach is robust against these failure modes, and our custom-trained extraction models keep extractions efficient and accurate.

Getting Started

For your most complex document extraction tasks, AI Extract Precision Mode is now available. Set the mode to precision when calling ai_extract function, or turn on the precision mode toggle in the Information Extraction UI on the Agents page:

Try AI Extract Precision Mode

Footnotes:

¹ We define accuracy as the fraction of extracted objects that match the ground-truth object. Scoring depends on type. Primitives (booleans, floats, integers, enums) use direct match. Strings try direct match first, then fuzzy match, then an LLM judge. Arrays are scored by finding the closest pairing between predicted and expected items, then averaging across pairs. Objects are scored per field by type, then averaged across all fields.