The 40-year-old database rule agents just broke: How LTAP unifies OLTP and OLAP workloads

Jonathan Katz has spent his career on both sides of a line that most of the data industry has simply learned to live with. As a longtime Postgres contributor and now a Senior Staff Product Manager at Databricks, he has watched operational and analytical systems operate as two separate worlds, bridged only by pipelines, copies, and compromise.

In this conversation, Jonathan walks through why that separation existed in the first place, why AI agents are the thing finally breaking it, and how LTAP closes the gap by rethinking where the two worlds meet: not in a single engine, but in a unified storage layer.

The assumption AI agents broke

The industry has lived with a strict separation between operational and analytical database systems for decades. Why is the rise of autonomous AI agents breaking that apart?

Jonathan Katz: There are two worlds of data. Operational data is what you touch when you're processing a credit card transaction or looking for fraud: very short, fast queries, looking at data line by line. Analytical data is what you've been accumulating for weeks, months, or years, and when you query it, you're looking across the entire dataset. Both are trying to return an answer as quickly as possible, but they go about it in completely different ways.

That difference isn't arbitrary. It comes down to physics. If you store your data in rows, that's how you return a single fast answer as quickly as possible. If you store it in columns, that's how you scan and aggregate across everything as quickly as possible. Also, by design, an analytical query can consume the entire resources of a massively parallelized system to get an answer over a very large dataset, while an operational query is designed to consume as few resources as possible while still returning an answer quickly. These lead to two very different ways to think about how you design and manage your data system, and require different kinds of optimizations.

Postgres, one of the most widely deployed databases on the planet, is built around rows because it's optimized for operational workloads. Analytical engines are built around columns for the opposite reason. Because of that fundamental physical difference, the two systems have always had to sit apart, and any time you wanted to analyze operational data, you had to ship it somewhere else.

The rise of AI agents has changed what we need from databases. It’s part of why Lakebase’s architecture exists in the first place: databases built to keep up with how agents actually work. For example, agents are tasked with looking for fraud or anomalies, and those events are happening in hundreds of milliseconds. But that system is also handling a huge volume of writes and short reads at the same time. A single agent might be smart enough to know what query it needs to run. A herd of agents can easily overwhelm an operational system if there aren’t guardrails in place.

What LTAP actually is

What is LTAP, and how does it actually work under the hood?

Jonathan Katz: LTAP, Lake Transactional/Analytical Processing, lets you run analytical queries directly against live operational data without moving that data anywhere and without putting load on the system serving your transactions. It does this by unifying transactional and analytical data in a single logical storage layer instead of forcing them through separate systems connected by a pipeline.

Under the hood, LTAP is only possible because of how Lakebase itself is architected: stateless, ephemeral compute that's fully decoupled from storage in the lake. That separation, inherited from Neon, means the durable storage layer can handle high-throughput writes and periodically flush them down to object storage for permanence, independent of whatever compute happens to be running against it at any given moment. Since that data was already being optimized for cloud storage, why not represent it in the same columnar format the Lakehouse already uses, so engines like Apache Spark and SQL can read it directly and get high-performance analytical reads without a second copy?

The hardest part was making sure nothing got lost in translation. Postgres has its own data types and encodings, and open formats like Iceberg and Delta have their own. We had to write the data out in a way that preserved the exact physical representation of the original Postgres data, without changing a single bit, and get it into a Parquet file. That's the piece that let us merge the operational and analytical representations of the same data into one. In practice, the storage layer runs in two tiers: a hotter tier that keeps data in row format for fast operational access, and a cooler tier that holds it in columnar format for analytical reads, so either side can get what it needs efficiently.

Why LTAP succeeds where HTAP stalled

HTAP tried to solve real-time analytics years ago and stalled out. Why does doing this at the lakehouse storage layer succeed where traditional HTAP failed?

Jonathan Katz: You can make HTAP systems work, but they're expensive. They're clunky, hard to run, and generally not open. What's different about the LTAP model is that you get serverless operational compute and serverless analytical compute as two separate things. You can tailor exactly how much compute you're using for each workload independently, rather than paying for one system trying to do both jobs at once. Storage is the cheap part of any data system. Compute is the expensive part.

That's the whole argument for unifying the storage layer instead of the engine: you get to keep the specialized, efficient engine for each job, and you only pay for compute where you actually need it, instead of running one expensive system that's trying to be good at everything at once.

What breaks when agents act on stale data

Walk me through a specific agentic workflow that breaks or degrades today because it's reading and acting on stale data. What actually goes wrong?

Jonathan Katz: Fraud detection is the clearest example. Credit card transactions are clearing in hundreds of milliseconds or less. If the agent responsible for catching fraud is working off a batch copy of the data that's minutes or hours old, it's simply too slow to catch anything before the transaction has already gone through. So you want that agent working directly against the operational system.

But the operational system is handling a constant stream of writes and short reads, and it wasn't built to also absorb heavy analytical queries. If the agent runs a query that scans a customer's entire purchase history to check for anomalies, that's an expensive query to run against a system that's optimized for the opposite kind of workload. It can degrade performance for every other transaction trying to clear at the same time. And a modern architecture usually needs data from both the operational and analytical sides to make a good decision, so the agent has to pull from both. One agent might handle that responsibly. A fleet of agents running similar queries at the same time can overwhelm the operational system fast if there's nothing governing how much load they're allowed to put on it.

Governance, openness and the enterprise case

How does Databricks specifically implement LTAP today, and how would you describe that to someone who already understands why HTAP falls short?

Jonathan Katz: Beyond the storage mechanics, the other major piece is the catalog. One of the real innovations of the Lakehouse was giving organizations a centralized, unified view of all their data: who has access to what, consistent policies across everything, so people can't read something like a social security number unless they're in a privileged group. That never really applied to operational systems, because operational systems were built as data silos from the start. The relationship between operational and analytical data used to be: you build a pipeline, you ship the data over, and after that, good luck. Nobody owned what happened to it downstream. LTAP flips that. It's all your data, in one unified storage model, under one catalog. You don't have to worry about operational data ever leaving a governance boundary just because someone needed to analyze it.

There's also a case for why this needs to be built on an open foundation. Postgres is closing in on being the third most sentimentally favored database on DB-Engines' rankings. That's not necessarily a measurement of adoption, but it's a strong signal of where things are headed, and it shows the value of flexibility and choice. Open source has been powering some of the most important systems in the world for decades. LTAP extends that same principle. Even within Postgres, your data is portable between Postgres systems, but you're still bound to Postgres. With LTAP's unified storage layer, you're not moving data around to get the right engine on it anymore. You're bringing the engine to the data.

The core shift: unified storage

If you had to describe the core shift LTAP represents in one sentence, however you'd frame it, how would you put it?

Jonathan Katz: The oversimplified version is unified storage. Tell that to someone from the analytics side and they get it almost immediately. Tell it to someone from the operational side and they may ask what you mean. But once you can bring the operational and analytical representations of the same data together without moving anything, without any bits changing, it collapses a lot of problems that used to feel unavoidable. I don't need to run pipelines anymore just to get data into a bronze or silver layer. I can start analyzing it the moment it's written. I'd also flip it around: this is less about inventing something new and more about bringing two worlds that never should have been separated back together. Data is just data. The more we can treat it that way, the easier it gets for people, and now agents, to work with it without everyone having to negotiate across a pipeline first.

Bringing both worlds back together

For forty years, the line between operational and analytical data held because the physics of storage demanded it. Agents are the first workload that can't tolerate the delay that line creates. LTAP doesn't try to erase the difference between a transaction and an analytical query. It removes the tax that used to come with running both against the same data. For a Data Architect evaluating whether an agentic workload actually needs this pattern, the test Jonathan describes is a useful one: if an agent's next decision depends on data that's still settling, on a system built to keep that data close and protected, the old pipeline-and-copy model isn't going to be fast enough. That's the specific problem LTAP was built to solve.

To find out more about LTAP, read From monolith to Lakebase to LTAP: rethinking the database from storage up.