Adopting Databricks and Unity Catalog Governance to Support ITGC Compliance

Introduction

The Sarbanes-Oxley Act of 2002 (SOX) is a U.S. federal law designed to enhance corporate governance, financial transparency, and the integrity of financial data for publicly traded companies and those planning to go public through an initial public offering (IPO). The primary goal of SOX is to protect investors by improving the accuracy of financial reporting which is achieved through enforced internal controls that strengthen the reliability of financial disclosures.

Information Technology General Controls (ITGCs) play an essential role in SOX compliance. They aim to ensure the security, integrity, and reliability of IT systems that support financial reporting, helping organizations prevent fraud and maintain the trust of the public and customers.

The Databricks Data Intelligence Platform, which includes Unity Catalog, offers comprehensive data governance, centralized access controls, auditing, and data lineage capabilities. Databricks offers organizations the tools to support the implementation of ITGCs by securing data, managing access, and supporting compliance across data and AI environments. Organizations are responsible for designing, implementing, and testing their controls and processes to align with their business and regulatory requirements.

Further, Databricks provides an ITGC best practices matrix that maps its platform features and capabilities, enabling organizations to align their technical controls with audit and compliance requirements efficiently.

Scope

Databricks empowers IT security and compliance professionals to meet key IT General Controls (ITGC) requirements, including access management, change control, backup and recovery, and IT operations monitoring. This overview’s targeted audience is for IT security and compliance professionals seeking practical strategies to implement and maintain robust IT General Controls (ITGCs) using Databricks.

What is Unity Catalog?

Databricks Unity Catalog is the industry’s only unified and open governance solution for a company’s data and AI across clouds and data platforms. Its foundation is the Databricks Data Intelligence Platform, which understands the uniqueness of your data and drives the most comprehensive and unified governance solution for all of your company’s data and AI. And it’s built on a lakehouse to be open, scalable, low cost, and high performance — the best of all worlds!

Security and Shared Responsibility Model

The Databricks shared responsibility model outlines the security and compliance obligations of Databricks, the cloud service provider (CSP), and the customer concerning the data and services on the Databricks Platform. Databricks maintains documentation for customers based on their cloud provider implementation: AWS, Azure, or GCP.

ITGC Compliance: Best Practices Matrix

As part of the shared responsibility model, Databricks operates as a hybrid platform SaaS provider. It is responsible for securing the platform's infrastructure, while customers are responsible for securing their data, access, and configurations within the platform. The platform provides a range of built-in and configurable security features, including encryption, access controls, audit logging, customer-managed keys, and network isolation.

For organizations utilizing Databricks, maintaining ITGC compliance is essential to ensuring the security, availability, and confidentiality of their data. To assist you in starting your ITGC compliance journey, Databricks has created a high-level summary guide that outlines best practices for customer capabilities.

We recommend that you review the best practices matrix below and collaborate with your internal and external security, compliance, and audit teams to align Databricks security controls with your organization's specific requirements.

Please refer to the mapping of ITGC Best Practices to Databricks’ Customer Capabilities here.

Note: This document and its accompanying control mapping guidance are provided for educational purposes only and may contain errors or omissions. We reserve the right to update these materials at any time, without prior notice. Readers are strongly advised to consult qualified technical and legal professionals to ensure proper control implementation and regulatory compliance.

Conclusion

Databricks provides a secure, robust platform offering customers various data governance and audit features to help them meet their ITGC compliance obligations.

Next Steps for ITGC Compliance with Databricks

• Explore Resources: Visit the Databricks Security and Trust Center to review security and compliance documentation, including the Databricks SOC 1 Type II reports.
• Leverage Unity Catalog: To meet ITGC requirements, use Unity Catalog for centralized governance, access controls, and data lineage tracking. To get started with Unity Catalog, follow our Unity Catalog guide for AWS, Azure, and GCP.
• Adopt ITGC Best Practices: Follow Databricks’ ITGC matrix and collaborate with your audit teams for compliance alignment.
• Engage Experts: Contact your Databricks account team to address Databricks’ specific technical details related to implementing ITGC-compliant solutions.

These steps will help you optimize Databricks’ capabilities to meet your compliance goals efficiently.