Abstract AlgorithmsAbstract Algorithms

  • Home
  • All Posts
  • All Series
  • About

Category

stateful processing

1 article

Stateful Aggregations in Spark Structured Streaming: mapGroupsWithState

TLDR: mapGroupsWithState gives each streaming key its own mutable state object, persisted in a fault-tolerant state store that checkpoints to object storage on every micro-batch. Where window aggregations assume fixed time boundaries, mapGroupsWithSt...

Apr 19, 2026•29 min read

Abstract Algorithms

Exploring the fascinating world of algorithms, data structures, and software engineering through clear explanations and practical examples.

Navigation

  • Home
  • All Posts
  • All Series
  • About

Series

  • Apache Spark Engineering
  • Python Programming

Popular Topics

  • #apache-spark
  • intermediate
  • performance
  • Python
  • Structured Streaming
  • big data

Author

Abstract Algorithms

Abstract Algorithms

@abstractalgorithms

1 followers on Hashnode

© 2026 Abstract Algorithms. All rights reserved.

Powered by Hashnode