Learn about Apache Spark’s Memory Model and Spark’s State in the Cloud
Since Apache Spark 1.6, as part of the Project Tungsten , we started an ongoing effort to substantially improve the memory and CPU...
Databricks invites Colleen Lewis to Speak about Diversity in the Workplace
First I'll start with the sad truth. The technology industry at large has taken many hits over the years for discriminatory practices and...
Learn about Apache Spark APIs and Best Practices
Since Apache Spark 1.3, Spark and its APIs have evolved to make them easier, faster, and smarter. The goal has been to unify...
Build, Scale, and Deploy Deep Learning Pipelines with Ease
At the Spark Summit in San Francisco in June , we announced an open-source project Deep Learning Pipelines . Deep Learning Pipelines provides...
Do your Streaming ETL at Scale with Apache Spark’s Structured Streaming
At the Spark Summit in San Francisco in June , we announced that Apache Spark’s Structured Streaming is marked as production-ready and shared...