Skip to main content
<
Page 33
>

Shell Oil Use Case: Parallelizing Large Simulations with Apache SparkR on Databricks

This blog post is a joint engineering effort between Shell’s Data Science Team ( Wayne W. Jones and Dennis Vallinga ) and Databricks...

Managing and Securing Credentials in Databricks for Apache Spark Jobs

June 20, 2017 by Jason Pohl in
Since Apache Spark separates compute from storage, every Spark Job requires a set of credentials to connect to disparate data sources. Storing those...

Analysing Metro Operations Using Apache Spark on Databricks

This is a guest blog from EY Advisory Data & Analytics team, who have been working with Sporveien in Oslo building a platform...

Databricks Serverless: Next Generation Resource Management for Apache Spark

As the amount of data in an organization grows, more and more engineers, analysts and data scientists need to analyze this data using...

Integrating Apache Spark with Cucumber for Behavioral-Driven Development

This is a guest blog from FIS Global One of the most difficult scenarios in data processing is ensuring that the data is...

Apache Spark Cluster Monitoring with Databricks and Datadog

This blog post is a joint effort between Caryl Yuhas, Databricks’ Solutions Architect, and Ilan Rabinovitch, Datadog’s ‎Director of Technical Community and Evangelism...

Top 5 Reasons for Choosing S3 over HDFS

At Databricks, our engineers guide thousands of organizations to define their big data and cloud strategies. When migrating big data workloads to the...

Entropy-based Log Redaction for Apache Spark on Databricks

May 30, 2017 by Weiluo Ren and Yu Peng in
This blog post is part of our series of internal engineering blogs on Databricks platform, infrastructure management, tooling, monitoring, and provisioning. We love...

Working with Nested Data Using Higher Order Functions in SQL on Databricks

View this notebook on Databricks Nested data types offer Databricks customers and Apache Spark users powerful ways to manipulate structured data. In particular...

Databricks Runtime 3.0 Beta Delivers Cloud Optimized Apache Spark

May 24, 2017 by Reynold Xin in
A major value Databricks provides is the automatic provisioning, configuration, and tuning of clusters of machines that process data. Running on these machines...