Collaborative Analytics on Databricks

Introduction

In our earlier blog, Enabling Business Users on Databricks, we explored how capabilities like conversational analytics, governed data access, and AI-powered applications empower business users to interact directly with data. But empowerment alone isn’t enough.

The real challenge for most financial institutions isn’t just enabling individual users - it’s enabling collaboration across teams. Clients, portfolio managers, operations specialists, and finance teams all rely on the same underlying data. Yet they often operate in separate systems, with fragmented workflows and manual handoffs in between. Insights move slowly. Data definitions drift. And leaders are left asking a familiar question:

“Are we all looking at the same numbers?”

The Databricks Data Intelligence Platform answers that question with a unified platform for data, analytics, AI, and operational workflows, allowing business users across the organization to collaborate on the same governed data. To see how this works in practice, let’s walk through a realistic workflow powered by Databricks SQL, UC Metric Views and Lakebase on Databricks.

The core tension: same data, different worlds

Imagine four professionals trying to answer questions about the same investment portfolio, each from a completely different perspective.

• Sarah, an actuary, wants to know if liability cash flows are aligned with asset durations.
• Dan, a portfolio manager, needs to confirm whether a client's portfolio is on-mandate and understand the current return vs. expected return.
• John, in operations, is reconciling IBOR and ABOR records and tracking down the biggest position breaks.
• Ben, in finance, needs to generate ledger entries and validate whether an adjustment will correctly close a book.

Each of them asks different questions. Each has different data access needs. Each uses different tools. Yet they all rely on the same underlying data: portfolios, positions, liabilities, and transactions.

Traditionally, organizations respond to this challenge with siloed systems—actuarial tools, portfolio platforms, reconciliation software, and ERP systems. Databricks replaces this fragmented model with a single governed data platform with unified semantics for every team.

The Databricks capability stack for business collaboration

For the technical team, the promise is one unified set of tools. For business users, the promise is fewer manual handoffs and more time spent on decisions, not data wrangling.

Here's how the stack maps to that promise:

• Talk to your data with Genie (Conversational Analytics). Business users ask questions in plain English and get answers backed by verified, governed data. No SQL required. No ticket to the data team. And through One Chat with intelligent routing, a user doesn't need to know which Genie space handles which domain! The system routes the query to the right context automatically.
• Seamless Handoff interface with Databricks Apps. Give business users rich, interactive interfaces where they can not only review reports but take action, adding notes, approving adjustments, and triggering downstream workflows all within a governed application layer.
• Low-latency serving layer with Lakebase. Serve as the transactional and operational data backbone, supporting the reconciliation checks, balance validations, and real-time writes that middle and back office workflows demand. It's the bridge between analytical insight and operational action.
• Drag and Drop with Lakeflow Designer. Enable data enrichment and transformation of Lakeflow Pipelines through a visual, low-code interface — letting teams like Sarah's enrich raw asset and liability data without waiting for engineering cycles.
• Strong data governance with Unity Catalog. Provide the isolation boundaries using row-level security, column masking, RBAC and ABAC policies, ensuring that Sarah's access to liability data and Ben's access to ledger entries are governed independently, even as they query the same underlying tables.
• Consistent term definitions with Unity Catalog Business Semantics. The same data, the right access and the same terminology across the organization.

And because the pace of AI model innovation isn't slowing down, Databricks’ model-agnostic architecture lets you adapt quickly, swapping in new models, embracing multimodal capabilities and spanning multicloud environments without re-architecting your data platform.

From intent to impact: a closed-loop business workflow

Let's make this concrete. The following scenario traces a single business intent — a request to adjust portfolio duration — from its origin in the actuary's analysis through to the final ledger entry in the back office.

Step 1: Strategy starts with the business (client)

Sarah, the actuary, is tasked with strategic asset allocation. Her job begins with a question: "Are our assets actually covering our liabilities in the right proportions, at the right durations?

She opens Genie on Databricks and asks, in plain language: "Are asset cash flows aligned with liability timing for matching analysis?" Genie queries the liability and asset tables, surfaces a duration mismatch, and presents it in an interactive dashboard. Sarah enriches the raw data using Lakeflow Designer Pipelines and concludes that the target duration must shift. She submits a formal request to change the mandate.

The message here: Strategy is data-driven from day one. The actuary's recommendation isn't built on a spreadsheet export from last Tuesday; it's built on the same live, governed data the rest of the organization uses.

Step 2: Front office translates strategy into action (portfolio management)

Dan, the portfolio manager, receives Sarah's request through a Databricks App. He can see the request in context, the underlying analysis, the duration gap and the proposed adjustment - all without leaving the platform.

From there, AI agents go to work. They pull the latest market data through an external MCP Server, run scenario models to understand the return and sector implications of the duration shift, and surface the trade-off analysis directly within Dan's workflow. Dan reviews the output, adjusts parameters, and translates the high-level intent — "shift duration by X years" into concrete portfolio changes: specific sector exposure adjustments, return targets, and position modifications that get communicated to the execution layer.

The message here: AI acts as a co-worker, not a black box. It accelerates the translation of strategic intent into actionable instruction while keeping the portfolio manager firmly in the decision-making seat.

Step 3: Middle office ensures operational integrity (operations / valuation)

Once portfolio changes are queued, John in operations takes over. His job is to make sure the Investment Book of Record (IBOR) and the Accounting Book of Record (ABOR) are reconciled.

John uses AI-powered reconciliation through Databricks Apps to review IBOR/ABOR records side by side. The system flags mismatches, surfaces root causes — whether a timing difference, a failed settlement, or a data mapping issue — and proposes corrective adjustments. Those adjustments are written directly into governed Lakebase tables, creating an auditable, timestamped record of every correction.

The message here: Controls and transparency are embedded in the workflow, not bolted on afterward. The middle office isn't chasing exceptions through email threads; they're resolving them in a governed, traceable environment.

Step 4: Back office closes the loop (investment office / finance)

Ben, in the back office, reviews the adjustment entries prepared by John's team. Using Databricks Apps and Lakebase, he approves the corrections, generates the corresponding general ledger entries, and runs a final risk review through AI/BI Dashboards, confirming that the portfolio's overall risk profile is within acceptable bounds following the mandate change.

Everything Ben sees, the portfolio positions, the reconciliation adjustments, the risk metrics, traces back to the same governed data platform that Sarah queried at the beginning of this workflow. There's no reconciliation between systems, because there's only one system.

The message here: Reporting, risk, and accounting operate on the same source of truth. The back office isn't catching up to the front office; it's completing the same loop, on the same data, in real time.

The executive takeaway

For financial services leaders, this pattern delivers four critical advantages:

1. One platform across the business, eliminating the integration tax of stitching together siloed tools.
2. AI embedded in business workflows, not siloed in data science, AI assists the people making day-to-day decisions more like a trusted co-worker.
3. Governed, real-time data from decision to ledger with Unity Catalog, ensuring that access, traceability, and compliance are never an afterthought.
4. Human + AI collaboration at every step, preserving human judgment and accountability while dramatically compressing the time from insight to action.

The story isn't about tools. It's about compressing strategy-to-execution cycles while strengthening controls. That's not just a technology story. That's a better way to run the business.

Ready to close the loop?

From actuary to finance, every decision deserves the same governed, real-time source of truth. Here's how to move forward:

• Try it out: Start your free Databricks trial today
• See it in action: Visit our demo center for product tours, videos and hands-on tutorials covering Lakeflow, Unity Catalog, AI/BI and more.
• Learn the basics: Get started with free Academy training
• Download: The Business Intelligence meets AI eBook

Ready to talk? Contact your Databricks account team to see how Databricks can transform your business users’ daily workflows.