Sponsored by: ChaosSearch | Expanding Log Analytics and Threat Hunting Natively In Databricks


TYPELightning Talk
TRACKData Lakehouse Architecture
INDUSTRYEnterprise Technology, Media and Entertainment, Financial Services
TECHNOLOGIESData Sharing, Apache Spark, SQL Analytics / BI / Visualizations
SKILL LEVELIntermediate

Databricks is the Data Intelligence Platform of choice — an evolution of lakehouse architecture that unifies data, analytics, and AI. However, some key raw data use cases are still addressed outside this platform. An example is log analytics - for observability, security analytics, and user insights. Hunting and troubleshooting are essential in log analytics, but require needle-in-a-haystack queries that leverage search and retrieve across full rows of live data with potentially wide, dynamic and nested schemas. We believe SREs and security analysts should be able to take advantage of the power of Databricks without leaving the ecosystem — using tools they already know via an Elastic API and OpenSearch Dashboards. In this session we’ll show you how adding ChaosSearch delivers these additional proactive data investigation capabilities to Spark and Delta Lake natively in Databricks, with unlimited data retention and dramatic cost savings vs. alternatives.


Ed Walsh