Automating Document Work with Long-Horizon AI Agents

Enterprises hold the majority of their knowledge in PDFs, slides, spreadsheets, and Word files. Agents cannot natively read or trust these sources without robust parsing, extraction, grounding, and retrieval. Teams that treat document context as first‑class infrastructure, rather than an add‑on to a chatbot, will unlock the next wave of automation.

From naive RAG to agentic engineering

The common 2023 RAG pattern, chunk plus embed plus top‑k retrieval into an LLM, helped with simple Q&A but faltered on multi‑step reasoning. Incremental fixes followed, including hybrid search, reranking, and query rewriting, which improved retrieval but kept intelligence confined to a brittle pipeline.

Agentic loops shifted the center of gravity. ReAct‑style orchestration enabled multi‑step planning, tool use, and self‑correction. As models improved at coding, code became a proxy for general computer use, letting agents navigate file systems, analyze spreadsheets, and compose workflows. Tooling standardized around protocols like MCP, so retrieval complexity moved from hand‑tuned pipelines into the agent harness itself. For repetitive operational work, teams often blend deterministic, graph‑based workflows with pockets of LLM reasoning to add guardrails.

Why document context is the bottleneck

Models remain blank slates without the right prompts, tools, connectors, and data. Complex tasks require richer context: task specs and runbooks, access to systems of record, web search, structured analytics from a lakehouse, and, critically, unstructured documents. The opportunity is an agent‑native document platform that serves both humans and agents, not a retrofit of human‑first storage systems.

Accuracy demands are high. General vision‑capable models frequently score around 60 to 70 percent on messy, real‑world documents, far from the 99 percent plus that operations require for domains like loan origination, KYC, or claims. Reaching that bar calls for a stack that includes specialized parsing, table and chart understanding, deep handling of PDF internals, and benchmarks that reflect true distributions.

Building an agent‑native document stack

A practical stack spans three layers:

• Parsing and understanding: Combine frontier models with fine‑tuned extractors for layout, tables, and charts, and add confidence scores and bounding boxes so outputs are auditable.
• Semantics and extraction: Convert heterogenous documents into structured outputs that downstream systems can trust, with grounding and attribution for human review or ETL into analytics platforms.
• Agentic retrieval: Replace naive top‑k with a toolbox an agent can compose, such as semantic search, BM25, grep, file navigation, and read and scroll operations.

The talk introduces LightParse, a free, open source PDF parser that outputs Markdown and is positioned as very fast for agent workflows. It does not replace specialized extraction for the hardest documents, but it is useful for rapid, accurate text representations when agents need to ingest files quickly.

Cost, orchestration, and the remaining gaps

Several gaps remain. Teams lack a truly agent‑native document format that is both human editable and machine friendly. Versioning, permissions, and audit for agent‑generated edits need to catch up so humans can review and roll back changes. Long‑running agents that act on event triggers, and batch UIs to operate agent swarms for high‑volume document workloads, are still evolving.

Cost optimization is now a discipline. Some tasks merit frontier reasoning, while high‑volume review benefits from cheaper models or open weights. Auto‑routing across model tiers, often served on the Databricks Platform, helps balance task fidelity against spend as horizons stretch from minutes to always on.

Turning document context into an operational advantage

Agent capability is compounding, yet reliable automation hinges on the depth and quality of the document context agents can access. Investing in parsing, grounded extraction, and agentic retrieval, along with thoughtful orchestration and cost routing, converts unstructured files into dependable fuel for long‑horizon agents.