Integrating Apache Spark with Cucumber for Behavioral-Driven Development
This is a guest blog from FIS Global One of the most difficult scenarios in data processing is ensuring that the data is...
Apache Spark Cluster Monitoring with Databricks and Datadog
This blog post is a joint effort between Caryl Yuhas, Databricks’ Solutions Architect, and Ilan Rabinovitch, Datadog’s Director of Technical Community and Evangelism...
Transactional Writes to Cloud Storage on Databricks
In another blog post published today , we showed the top five reasons for choosing S3 over HDFS. With the dominance of simple...
Top 5 Reasons for Choosing S3 over HDFS
At Databricks, our engineers guide thousands of organizations to define their big data and cloud strategies. When migrating big data workloads to the...
Entropy-based Log Redaction for Apache Spark on Databricks
This blog post is part of our series of internal engineering blogs on Databricks platform, infrastructure management, tooling, monitoring, and provisioning. We love...