pyspark.streaming.DStream.window

DStream.window(windowDuration: int, slideDuration: Optional[int] = None) → pyspark.streaming.dstream.DStream[T]

Return a new DStream in which each RDD contains all the elements in seen in a sliding window of time over this DStream.

Parameters

windowDurationint

width of the window; must be a multiple of this DStream’s batching interval

slideDurationint, optional

sliding interval of the window (i.e., the interval after which the new DStream will generate RDDs); must be a multiple of this DStream’s batching interval

pyspark.streaming.DStream.updateStateByKey pyspark.streaming.kinesis.KinesisUtils.createStream