Approximate Algorithms in Apache Spark: HyperLogLog and Quantiles
Introduction Apache Spark is fast, but applications such as preliminary data exploration need to be even faster and are willing to sacrifice some...
<
Page 11
Faster Stateful Stream Processing in Apache Spark Streaming
Many complex stream processing pipelines must maintain state across a period of time. For example, if you are interested in understanding user behavior...
Diving into Apache Spark Streaming's Execution Model
With so many distributed stream processing engines available, people often ask us about the unique benefits of Apache Spark Streaming . From early...
New Visualizations for Understanding Apache Spark Streaming Applications
Earlier, we presented new visualizations introduced in Apache Spark 1.4.0 to understand the behavior of Spark applications. Continuing the theme, this blog highlights...
Improvements to Kafka integration of Spark Streaming
Apache Kafka is rapidly becoming one of the most popular open source stream ingestion platforms. We see the same trend among the users...
Introducing Streaming k-means in Apache Spark 1.2
Many real world data are acquired sequentially over time, whether messages from social media users, time series from wearable sensors, or — in...
Improved Fault-tolerance and Zero Data Loss in Apache Spark Streaming
Real-time stream processing systems must be operational 24/7, which requires them to recover from all kinds of failures in the system. Since its...
Apache Spark as a platform for large-scale neuroscience
The brain is the most complicated organ of the body, and probably one of the most complicated structures in the universe. It’s millions...
Apache Spark 1.1: The State of Spark Streaming
With Apache Spark 1.1 recently released, we’d like to take this occasion to feature one of the most popular Spark components - Spark...
Announcing Apache Spark 1.1
Today we’re thrilled to announce the release of Apache Spark 1.1! Apache Spark 1.1 introduces many new features along with scale and stability improvements. This post will introduce some key features of Apache Spark 1.1 and provide context on the priorities of Spark for this and the next release.