Skip to main content
Anish Shrigondekar

Anish Shrigondekar

Anish Shrigondekar's posts

Screenshot of Python code snippet for automatic schema evolution

Data Engineering

December 1, 2025/11 min read

From Events to Insights: Complex State Processing with Schema Evolution in transformWithState

Continuous Environmental Monitoring Using the New transformWithState API

Solutions

July 30, 2025/9 min read

Continuous Environmental Monitoring Using the New transformWithState API

Apache Spark 4.0

Open Source

May 28, 2025/10 min read

Introducing Apache Spark 4.0

The Evolution of Arbitrary Stateful Stream Processing in Spark

Solutions

May 7, 2025/12 min read

The Evolution of Arbitrary Stateful Stream Processing in Spark

Introducing transformWithState in Apache Spark™ Structured Streaming

Announcements

February 24, 2025/6 min read

Introducing transformWithState in Apache Spark™ Structured Streaming

Introducing Easier Change Data Capture in Apache Spark™ Structured Streaming

Solutions

January 27, 2025/6 min read

Announcing Simplified State Tracking in Apache Spark™ Structured Streaming

UA

Data Engineering

February 28, 2024/4 min read

A Deep Dive into the Latest Performance Improvements of Stateful Pipelines in Apache Spark Structured Streaming

stream

Data Engineering

February 27, 2024/5 min read

Performance Improvements for Stateful Pipelines in Apache Spark Structured Streaming

Announcing Support for Enhanced Fan-Out for Kinesis on Databricks

Solutions

January 30, 2023/8 min read

Announcing Support for Enhanced Fan-Out for Kinesis on Databricks