Graph-Powered Observability Data Analysis in Databricks With Credential Vending

Observability data — logs, metrics, and traces — captures the complex interactions within modern distributed systems. A graph query engine on top of Databricks enables complex traversal of massive observability data, helping users trace service dependencies, analyze upstream/downstream impacts, and uncover recurring error patterns, making it easier to diagnose issues and optimize system performance.

A critical challenge in handling observability data is managing dynamic RBAC for the sensitive system telemetry. This session explains how Coinbase leverages credential vending, a method for issuing short-lived credentials to enable fine-grained, secure access to observability data stored in Databricks without long-lived secrets.

Key takeaways:

• Querying Databricks tables as graph structures without ETLing data out
• Secure access management with credential vending
• Practical graph-based incident analysis solution at Coinbase, with insights on how PuppyGraph enables this application

Audio for this session is delivered in the conference mobile app, you must bring your own headphones to listen.