Why Vector Stores Are Not Enough: Using Lakebase as a Durable Memory Layer for Autonomous Agents

We're often told that Vector Databases are the "memory" for AI Agents. But vector search only gives you a probabilistic similarity score. In enterprise workflows, "probably" isn't good enough. Agents need deterministic, transactional state—and they need the ability to compound their knowledge over time.

In this session, we'll move beyond stateless "goldfish" chatbots and build a truly self-improving agent system using Databricks Apps, Databricks Jobs, and Lakebase Autoscaling.

We'll demonstrate how to build a production-ready "Stateful Worker" agent, featuring asynchronous batch "dreaming" distillation jobs that allow your agents to distill insights and get smarter overnight. To ground these concepts, we'll walk through the architecture from three critical perspectives:

• The Agent User Experience: How can we ensure an agent knows a user's role, preferences, and past context? We will demo the stateless stateful agents on Databricks Apps and see how users can benefit from user and agent scoped memory.
• The Database Admin Experience: How do we audit what an agent is learning? We'll explore Lakebase's intuitive UI/SQL, understand the Agent's thinking with observability tools, and leverage database branching and Point-in-Time Restore (PITR) to create isolated simulation sandboxes where agents can test destructive actions (e.g., "Delete User") safely.
• The Agent Developer Experience: How can I go from nothing to a fully deployed, memory-enabled enterprise agent in minutes, while still maintaining full control over the underlying code? We'll be showing how simple it is to author custom code agents with full integration with Databricks resources.

Attendees will leave with a reusable architecture, deployment templates, and a concrete framework to build high-performing, autonomous, and secure agents in production.