What is Databricks Runtime?
Explore the optimized compute engine that powers faster Spark performance, enterprise security, and simplified cluster management
Archived. This article has not been updated since the publish date above. The dynamic nature of information means that previously accurate content can become outdated or even obsolete over time. Readers are advised to exercise due diligence and cross-check any information found in this blog post before making decisions or adopting any practices based on said information.
- Databricks Runtime is the optimized software stack that runs on Databricks clusters and includes Apache Spark plus performance enhancements and preinstalled libraries.
- Different runtime versions are tuned for workloads such as SQL, data engineering, machine learning or streaming so jobs run reliably and efficiently.
- Using Databricks Runtime lets teams spend less time on environment setup and more time building data and AI solutions on a managed, high performance platform.
Databricks Runtime is the set of software artifacts that run on the clusters of machines managed by Databricks. It includes Spark but also adds a number of components and updates that substantially improve the usability, performance, and security of big data analytics. The primary differentiations are:
- Better Performance with DBIO: The Databricks I/O module, or DBIO, leverages the vertically integrated stack to significantly improve the performance of Spark in the cloud.
- Stronger Security with DBES: The Databricks Enterprise Security, or DBES, module adds features such as data encryption at rest and in motion, fine-grained data access control, and auditing to satisfy standard compliance (e.g. HIPAA, SOC2) and the most stringent security requirements as one would expect of large enterprises.
- Significantly lower operational complexity: With features such as auto-scaling of compute resources and local storage, we put Spark on "autopilot" and markedly reduce the operational complexity and management cost.
- Rapid releases and early access to new features: Compared to upstream open source releases, Databricks' SaaS offering facilitates quicker release cycles, offering our customers the latest features and bug fixes that are not yet available in open source releases.
The agentic AI playbook for the enterprise
DATABRICKS RUNTIME OUTPERFORMS OTHER COMPUTE ENGINES
Additional Resources
Get the latest posts in your inbox
Subscribe to our blog and get the latest posts delivered to your inbox.